-- Logs begin at Tue 2022-08-02 09:15:55 UTC, end at Tue 2022-08-02 10:55:10 UTC. -- Aug 02 09:44:54.465253 dv-cinder-843277-8-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 02 09:44:54.479041 dv-cinder-843277-8-iscsi devstack@keystone.service[80143]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 02 09:44:54.480067 dv-cinder-843277-8-iscsi devstack@keystone.service[80143]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue Aug 2 09:44:54 2022] *** Aug 02 09:44:54.480067 dv-cinder-843277-8-iscsi devstack@keystone.service[80143]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 02 09:44:54.480067 dv-cinder-843277-8-iscsi devstack@keystone.service[80143]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 02 09:44:54.480067 dv-cinder-843277-8-iscsi devstack@keystone.service[80143]: nodename: dv-cinder-843277-8-iscsi Aug 02 09:44:54.480067 dv-cinder-843277-8-iscsi devstack@keystone.service[80143]: machine: x86_64 Aug 02 09:44:54.480067 dv-cinder-843277-8-iscsi devstack@keystone.service[80143]: clock source: unix Aug 02 09:44:54.480067 dv-cinder-843277-8-iscsi devstack@keystone.service[80143]: pcre jit disabled Aug 02 09:44:54.480067 dv-cinder-843277-8-iscsi devstack@keystone.service[80143]: detected number of CPU cores: 4 Aug 02 09:44:54.480067 dv-cinder-843277-8-iscsi devstack@keystone.service[80143]: current working directory: / Aug 02 09:44:54.480067 dv-cinder-843277-8-iscsi devstack@keystone.service[80143]: detected binary path: /usr/bin/uwsgi-core Aug 02 09:44:54.480675 dv-cinder-843277-8-iscsi devstack@keystone.service[80143]: your processes number limit is 47680 Aug 02 09:44:54.480675 dv-cinder-843277-8-iscsi devstack@keystone.service[80143]: your memory page size is 4096 bytes Aug 02 09:44:54.480675 dv-cinder-843277-8-iscsi devstack@keystone.service[80143]: detected max file descriptor number: 2048 Aug 02 09:44:54.480675 dv-cinder-843277-8-iscsi devstack@keystone.service[80143]: lock engine: pthread robust mutexes Aug 02 09:44:54.480675 dv-cinder-843277-8-iscsi devstack@keystone.service[80143]: thunder lock: enabled Aug 02 09:44:54.480675 dv-cinder-843277-8-iscsi devstack@keystone.service[80143]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 02 09:44:54.480675 dv-cinder-843277-8-iscsi devstack@keystone.service[80143]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 02 09:44:54.519650 dv-cinder-843277-8-iscsi devstack@keystone.service[80143]: Python main interpreter initialized at 0x559de411d9c0 Aug 02 09:44:54.519650 dv-cinder-843277-8-iscsi devstack@keystone.service[80143]: python threads support enabled Aug 02 09:44:54.519650 dv-cinder-843277-8-iscsi devstack@keystone.service[80143]: your server socket listen backlog is limited to 100 connections Aug 02 09:44:54.519650 dv-cinder-843277-8-iscsi devstack@keystone.service[80143]: your mercy for graceful operations on workers is 90 seconds Aug 02 09:44:54.520058 dv-cinder-843277-8-iscsi devstack@keystone.service[80143]: mapped 403077 bytes (393 KB) for 2 cores Aug 02 09:44:54.520135 dv-cinder-843277-8-iscsi devstack@keystone.service[80143]: *** Operational MODE: preforking *** Aug 02 09:44:54.520397 dv-cinder-843277-8-iscsi devstack@keystone.service[80143]: *** uWSGI is running in multiple interpreter mode *** Aug 02 09:44:54.520397 dv-cinder-843277-8-iscsi devstack@keystone.service[80143]: spawned uWSGI master process (pid: 80143) Aug 02 09:44:54.520616 dv-cinder-843277-8-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 02 09:44:54.522177 dv-cinder-843277-8-iscsi devstack@keystone.service[80143]: spawned uWSGI worker 1 (pid: 80153, cores: 1) Aug 02 09:44:54.522177 dv-cinder-843277-8-iscsi devstack@keystone.service[80143]: spawned uWSGI worker 2 (pid: 80154, cores: 1) Aug 02 09:44:54.522336 dv-cinder-843277-8-iscsi devstack@keystone.service[80143]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 02 09:45:00.073456 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 02 09:45:00.073456 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: from cryptography.utils import int_from_bytes Aug 02 09:45:00.074259 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 02 09:45:00.074259 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: from cryptography.utils import int_from_bytes Aug 02 09:45:00.074373 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 02 09:45:00.074373 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: from cryptography.utils import int_from_bytes Aug 02 09:45:00.074373 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 02 09:45:00.074373 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: from cryptography.utils import int_from_bytes Aug 02 09:45:00.125394 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 02 09:45:00.125606 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 02 09:45:00.125766 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] command line args: [] {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 02 09:45:00.125868 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 02 09:45:00.125952 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 02 09:45:00.126028 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 02 09:45:00.126144 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] command line args: [] {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 02 09:45:00.126224 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] ================================================================================ {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 02 09:45:00.126415 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 02 09:45:00.126498 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] admin_token = **** {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:45:00.126603 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] ================================================================================ {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 02 09:45:00.126692 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] config_dir = [] {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:45:00.126859 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:45:00.126937 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] admin_token = **** {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:45:00.127046 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] config_source = [] {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:45:00.127125 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] config_dir = [] {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:45:00.127228 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] debug = True {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:45:00.127315 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:45:00.127429 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:45:00.127683 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] config_source = [] {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:45:00.127683 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] debug = True {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:45:00.127810 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:45:00.127876 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:45:00.127965 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:45:00.128027 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:45:00.128192 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:45:00.128376 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:45:00.128477 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:45:00.128585 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:45:00.128689 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] list_limit = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:45:00.128789 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:45:00.128885 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] log_config_append = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:45:00.128997 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] list_limit = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:45:00.129100 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:45:00.129201 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] log_config_append = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:45:00.129297 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] log_dir = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:45:00.129397 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:45:00.129492 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] log_file = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:45:00.129596 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] log_dir = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:45:00.129596 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] log_file = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:45:00.129774 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:45:00.129875 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:45:00.129976 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:45:00.130071 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:45:00.130176 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:45:00.130284 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:45:00.130385 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:45:00.130483 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:45:00.130585 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:45:00.130687 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:45:00.130797 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:45:00.130911 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:45:00.131015 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:45:00.131118 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:45:00.131221 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:45:00.131334 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:45:00.131438 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:45:00.131539 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:45:00.131636 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:45:00.131736 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:45:00.131837 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:45:00.131966 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:45:00.132071 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:45:00.132183 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:45:00.132319 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:45:00.132489 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:45:00.132646 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:45:00.132772 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:45:00.132857 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:45:00.133127 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:45:00.133302 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.64/identity {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:45:00.133489 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] publish_errors = False {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:45:00.133643 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.64/identity {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:45:00.133753 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:45:00.133856 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] publish_errors = False {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:45:00.133965 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:45:00.134063 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:45:00.134160 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:45:00.134255 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:45:00.134624 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:45:00.134805 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:45:00.135004 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:45:00.135188 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:45:00.135361 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] standard_threads = False {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:45:00.135534 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:45:00.135653 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:45:00.135888 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] standard_threads = False {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:45:00.136011 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:45:00.136135 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:45:00.136279 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:45:00.136279 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] use_journal = False {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:45:00.136514 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:45:00.136647 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] use_json = False {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:45:00.136766 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:45:00.136882 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] use_stderr = False {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:45:00.137014 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] use_journal = False {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:45:00.137150 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] use_syslog = False {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:45:00.137268 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] use_json = False {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:45:00.137268 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] use_stderr = False {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:45:00.137502 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:45:00.137678 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] use_syslog = False {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:45:00.137807 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.137984 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 09:45:00.138118 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.138236 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.138387 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.138512 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.138621 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.138734 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.138862 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.138980 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.139094 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.139216 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.139332 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.139446 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.139569 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.139692 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.139805 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.139919 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.140041 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.140158 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.140647 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.140773 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.140888 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.141003 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.141118 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.141231 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.141373 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.141490 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.141616 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.141735 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.141857 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.141969 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.142201 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.142319 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.142420 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.142518 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.142623 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.142742 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.142861 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.142975 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.143081 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.143184 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.143299 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.143421 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.143551 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.143674 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.143786 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.143899 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.144015 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.144132 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.144338 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.144464 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.144575 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.144681 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.144794 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.144901 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.145031 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.145149 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.145255 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.145361 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.145470 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.146078 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.146189 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.146328 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.146460 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.146578 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.146689 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.146807 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.146919 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.147030 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.147165 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.147273 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.147396 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.147518 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.147632 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.147741 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.147850 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.147959 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.148072 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.148326 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.148442 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.148552 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.148661 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.148772 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.148888 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.149009 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.149125 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.149233 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.149342 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.149453 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.149561 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.149688 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.149799 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.149919 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.150034 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.150143 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.150262 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.150441 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.150557 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.150665 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.150812 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.150961 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.151097 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.151248 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.151389 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.151528 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.151680 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.151830 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.152026 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.152187 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.152374 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.152519 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.152702 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.152822 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.152931 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.153051 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.153163 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.153273 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.153387 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.153520 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.153632 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.153761 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.153885 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.153996 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.154174 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.154306 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.154421 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.154584 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.154705 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.154828 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.154940 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.155050 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.155226 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.155348 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.155457 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.155580 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.155694 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.155804 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.155916 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.156083 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.156192 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.156343 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.156453 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.156569 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.156733 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.156851 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.156976 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.157105 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.157217 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.157327 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.157440 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.157549 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.157670 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.157784 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.157894 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.158014 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.158143 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.158269 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.158403 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.158530 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.158713 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.158838 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.158935 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.159011 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.159094 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.159174 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.159254 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.159328 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.159407 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.159481 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.159561 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.159635 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.159763 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.159840 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.159912 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.159990 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.160142 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.160219 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.160347 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.160422 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.160494 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.160566 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.160651 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.160739 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.160824 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.160898 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.160970 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.161042 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.161119 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.161231 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.161322 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.161922 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.161997 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.162078 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.162153 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.162272 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.162409 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.162531 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.162644 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.162755 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.162865 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.162977 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.163103 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.163224 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.163355 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.163467 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.163580 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.163696 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.163821 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] auth.external = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.163936 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.164045 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.164156 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] auth.external = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.164294 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.164427 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.164579 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.164701 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.164824 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] auth.password = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.164824 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] auth.token = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.165036 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.165162 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.165291 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] auth.password = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.165411 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.165521 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] auth.token = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.165630 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.165741 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.165852 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.165964 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.166100 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.166225 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.166339 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.166451 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.166562 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.166676 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.166798 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] credential.caching = True {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.166914 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.167031 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.167150 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.167258 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.167369 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] credential.caching = True {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.167490 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.167589 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.167696 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.167838 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.167961 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.168084 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.168208 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.168405 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.168519 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.168632 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.168729 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.168834 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.168958 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.169069 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.169179 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.169289 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.169400 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.169521 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.169521 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.169762 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.169762 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.169956 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.170069 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.170165 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.170287 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.170401 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.170511 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] federation.caching = True {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.170630 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.170736 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.170845 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.170975 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.171090 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] federation.caching = True {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.171202 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.171316 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.171442 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.171556 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.171683 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.171796 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.171906 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.172020 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.172133 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.172264 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.172403 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.172513 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.172621 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.172733 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.172845 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.172957 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.173082 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.173208 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.173325 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] identity.caching = True {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.173439 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.173551 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.173665 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.173887 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.174005 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] identity.caching = True {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.174115 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.174235 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.174342 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.174504 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.174632 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.174745 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.174856 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.174981 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.175143 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.175316 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.175434 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.175547 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.175547 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.175752 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.175752 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.175993 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.176113 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.176221 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.176376 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.176487 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.176599 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.176735 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.176867 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.176991 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.177101 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.177210 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.177210 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.177425 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.177539 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.177653 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.177782 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.177907 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.178015 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.178137 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.178137 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.178343 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.178455 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.178574 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.178699 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.178822 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.178931 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.179042 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.179162 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.179274 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.179387 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.179524 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.179627 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.179719 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.179830 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.179941 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.180060 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.180183 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.180320 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.180436 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.180550 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.180657 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.180768 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.180898 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.181011 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.181125 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.181236 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.181356 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.181463 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.181572 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.181683 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.181804 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.181924 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.182022 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.182149 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.182277 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.182400 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.182524 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.182639 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.182769 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.182885 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.183011 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.183124 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.183646 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.183784 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.183896 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.184005 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.184132 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.184285 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.184403 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.184518 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.184634 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.184765 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.184894 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.185017 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.185134 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.185249 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.185365 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.185480 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.185609 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.185726 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.185842 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.185957 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] ldap.user = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.186078 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.186200 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.186333 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.186456 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.186576 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] ldap.user = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.186695 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.186819 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.186949 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.187083 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.187203 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.187324 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.187444 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.187562 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.187700 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.187821 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.187941 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.188060 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.188178 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.188374 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.188508 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.188628 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.188742 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.188923 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.189046 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.189166 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.189296 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.189413 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.189530 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.189647 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.189764 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.189879 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.189879 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.190101 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.190219 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.190336 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.190336 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.190551 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.190679 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.190799 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.190916 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.191032 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.191155 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.191278 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.191404 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.191515 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.191630 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.191749 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.191869 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.191982 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.192101 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.192213 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.192415 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.192556 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.192669 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.192772 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.192896 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.193007 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.193116 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.193227 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.193335 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.193447 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.193587 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.193717 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.193844 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.193969 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.194097 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.194227 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.194373 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.194506 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.194636 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.194770 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.194901 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.195032 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.195177 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] resource.caching = True {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.195308 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.195438 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.195573 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.195704 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.195835 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] resource.caching = True {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.195979 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.196110 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.196219 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.196393 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.196516 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.196597 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.196686 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.196766 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.196839 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.196911 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.196984 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.197056 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.197206 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.197283 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.197357 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] role.caching = True {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.197430 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.197502 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] role.driver = sql {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.197502 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.197647 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.197722 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.197796 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] role.caching = True {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.197869 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.197942 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] role.driver = sql {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.198015 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.198096 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.198171 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.198244 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.198316 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.198388 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.198460 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.198541 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.198616 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.198616 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.198747 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.198820 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.198820 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.199020 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.199095 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.199095 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.199227 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.199299 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.199385 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.199506 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.199584 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.199657 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.199657 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.199787 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.199868 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.199868 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.199991 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.200063 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.200063 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.200193 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.200293 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.200360 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.200420 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.200420 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.200526 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.200585 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.200585 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.200585 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.200762 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.200834 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.201063 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.201169 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.201315 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.201555 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.201634 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.201794 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.201993 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.202073 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.202252 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] token.caching = True {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.202418 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.202498 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.202725 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.202858 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.202967 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.203227 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.203307 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.203477 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.203694 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.203774 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.203952 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.204161 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.204284 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.204461 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.204657 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.204738 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.204926 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.205104 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.205184 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.205395 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.205538 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.205644 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.205875 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.205954 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.206119 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.206371 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.206452 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.206662 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] database.connection = **** {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.206850 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.206939 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.207146 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.207316 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.207397 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.207623 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.207728 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.207861 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.208643 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.208733 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.208886 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.209096 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.209228 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.209359 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.209534 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] token.caching = True {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.209664 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.209802 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.209926 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.210050 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.210272 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.210363 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.210510 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.210585 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.210756 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.210825 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.210987 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.211065 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.211230 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.211300 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.211468 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.211536 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.211641 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 02 09:45:00.211777 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.212042 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.213623 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.213623 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.213623 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.213623 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.213623 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.213623 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.214260 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.214260 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.214479 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] database.connection = **** {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.214579 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.214774 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.215014 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.215263 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.215504 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.215745 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.215984 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.216225 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.216492 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.216732 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.216979 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.217216 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.217465 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.217702 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.217949 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.218181 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.218425 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.218669 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 09:45:00.218820 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 02 09:45:00.241931 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80154) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 09:45:00.242157 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80154) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 09:45:00.242367 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80154) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 09:45:00.244481 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80153) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 09:45:00.244713 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80153) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 09:45:00.244937 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80153) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 09:45:00.251125 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80154) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 09:45:00.253446 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80153) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 09:45:00.281959 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80154) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 09:45:00.282190 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80154) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 09:45:00.283739 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80153) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 09:45:00.283971 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80153) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 09:45:00.340374 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80154) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 09:45:00.342021 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80153) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 09:45:00.393119 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80154) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 09:45:00.393303 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80154) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 09:45:00.394601 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80153) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 09:45:00.394828 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80153) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 09:45:00.460263 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80153) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 09:45:00.460503 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80153) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 09:45:00.460598 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80154) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 09:45:00.460726 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80153) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 09:45:00.460838 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80154) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 09:45:00.460947 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80153) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 09:45:00.461035 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80154) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 09:45:00.461167 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80153) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 09:45:00.461253 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80154) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 09:45:00.461383 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80153) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 09:45:00.461470 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80154) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 09:45:00.461597 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80154) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 09:45:00.485974 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80153) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 09:45:00.486165 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80154) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 09:45:00.486263 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80153) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 09:45:00.486350 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80154) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 09:45:00.486432 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80153) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 09:45:00.486509 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80154) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 09:45:00.486625 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80153) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 09:45:00.486722 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80154) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 09:45:00.486844 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80153) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 09:45:00.486929 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80154) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 09:45:00.487061 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80153) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 09:45:00.487147 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80154) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 09:45:00.487279 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80153) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 09:45:00.487374 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80154) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 09:45:00.499384 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80154) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 09:45:00.499552 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80153) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 09:45:00.512546 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80154) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 09:45:00.512726 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80153) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 09:45:00.512837 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80154) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 09:45:00.512944 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80153) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 09:45:00.513011 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80154) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 09:45:00.513074 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80153) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 09:45:00.513198 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80154) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 09:45:00.513305 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80153) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 09:45:00.513415 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80154) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 09:45:00.513489 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80153) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 09:45:00.513627 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80154) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 09:45:00.513712 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80153) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 09:45:00.513853 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80154) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 09:45:00.514049 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80153) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 09:45:00.531053 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.531243 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 02 09:45:00.531419 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.531584 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.531694 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 02 09:45:00.531776 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 02 09:45:00.532107 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.532292 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.532374 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 02 09:45:00.532451 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.532543 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.532624 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.532746 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.532832 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.533004 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.533281 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.542933 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.543207 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.543481 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.543767 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.543853 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.544079 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.544369 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.544648 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.550765 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80154) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 09:45:00.551497 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80153) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 09:45:00.556174 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80154) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 09:45:00.556487 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.556772 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.557029 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80153) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 09:45:00.557126 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.557317 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.557402 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.557578 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.557670 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.557847 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.557977 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.558116 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.558206 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.558387 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.558525 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.558645 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.558740 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.558912 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.559036 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.559172 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.559441 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.559705 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.581648 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.582572 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.583146 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80154) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 09:45:00.583429 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.584067 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80153) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 09:45:00.584448 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.588011 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80154) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 09:45:00.588312 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.588597 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.589336 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80153) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 09:45:00.589613 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.589897 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.595158 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80154) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 09:45:00.595469 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.596495 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80153) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 09:45:00.596855 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.600140 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80154) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 09:45:00.600487 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.600680 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.600961 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.601240 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.601523 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.601802 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.601954 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80153) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 09:45:00.602104 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.602253 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.602771 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.603095 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.603368 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.603648 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.603921 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.604195 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.618676 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.618953 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.620535 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.620879 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.622877 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80154) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 09:45:00.625250 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80153) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 09:45:00.626021 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.626317 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.628386 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.628678 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.630734 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80154) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 09:45:00.633149 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80153) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 09:45:00.633940 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80154) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 09:45:00.636311 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80153) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 09:45:00.637139 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.637433 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.637727 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.638017 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.638309 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.638601 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.639514 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.639804 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.640086 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.640397 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.640676 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.640967 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.654634 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80154) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 09:45:00.654863 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.655129 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.655391 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.656813 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80153) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 09:45:00.657105 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.657368 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.657628 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.662762 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.664318 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.664597 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.665084 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.666600 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.666875 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.667375 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80154) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 09:45:00.667660 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.667960 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.668265 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.668559 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.669673 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80153) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 09:45:00.669956 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.670251 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.670541 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.670819 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.680558 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80154) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 09:45:00.680851 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.681133 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.681419 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.681700 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.681970 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.682265 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.682847 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80153) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 09:45:00.683131 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.683414 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.683690 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.683965 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.684232 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.684543 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.697874 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80154) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 09:45:00.700187 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80153) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 09:45:00.700836 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80154) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 09:45:00.703178 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80153) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 09:45:00.703768 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.706119 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.706436 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.707957 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80154) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 09:45:00.708307 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.708592 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.708809 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.710299 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80153) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 09:45:00.710593 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.710904 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.715030 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.716633 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80154) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 09:45:00.717418 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.718996 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80153) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 09:45:00.719522 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.719812 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.720102 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.720407 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.721906 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.722191 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.722476 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.722751 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.729222 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80154) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 09:45:00.729515 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.729800 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.731680 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80153) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 09:45:00.731976 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.732280 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.736608 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80154) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 09:45:00.736892 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.737185 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.737473 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.737749 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.738032 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.738305 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.738586 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.738966 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.739051 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80153) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 09:45:00.739270 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.739953 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.740087 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.740181 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.740377 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.740463 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.740652 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.740792 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.740911 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.740995 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.741185 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.741447 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.741827 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.742118 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.742396 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.742655 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.742924 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.743182 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 09:45:00.774750 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x559de411d9c0 pid: 80154 (default app) Aug 02 09:45:00.777131 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x559de411d9c0 pid: 80153 (default app) Aug 02 09:45:00.778284 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1df54e-dedd-478c-aa3d-65739a09dc9a None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:00.778576 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1df54e-dedd-478c-aa3d-65739a09dc9a None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:00.778855 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1df54e-dedd-478c-aa3d-65739a09dc9a None None] PATH_INFO: `/v3` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:00.783219 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1/1] 10.222.0.64 () {50 vars in 791 bytes} [Tue Aug 2 09:45:00 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:45:08.566010 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fbf6da-e58a-46ca-8e79-1422ece08e38 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:08.566584 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fbf6da-e58a-46ca-8e79-1422ece08e38 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:08.566584 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fbf6da-e58a-46ca-8e79-1422ece08e38 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:08.570996 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1/2] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:45:08 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 09:45:08.576495 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c678105-2c1d-45c8-a0e8-430f9f117410 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:08.576917 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c678105-2c1d-45c8-a0e8-430f9f117410 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:08.577059 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c678105-2c1d-45c8-a0e8-430f9f117410 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:08.648790 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: INFO dbcounter [None req-4c678105-2c1d-45c8-a0e8-430f9f117410 None None] Registered counter for database keystone Aug 02 09:45:08.667899 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG oslo_db.sqlalchemy.engines [None req-4c678105-2c1d-45c8-a0e8-430f9f117410 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80154) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 02 09:45:08.674221 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writer thread running {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 02 09:45:08.859087 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG passlib.handlers.bcrypt [None req-4c678105-2c1d-45c8-a0e8-430f9f117410 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80154) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 02 09:45:08.859422 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG passlib.handlers.bcrypt [None req-4c678105-2c1d-45c8-a0e8-430f9f117410 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80154) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 02 09:45:08.916762 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-4c678105-2c1d-45c8-a0e8-430f9f117410 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:45:08.918378 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-4c678105-2c1d-45c8-a0e8-430f9f117410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:45:09.054152 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 2/3] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:45:08 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 479 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Aug 02 09:45:09.196021 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fbf6da-e58a-46ca-8e79-1422ece08e38 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:09.264754 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: INFO dbcounter [None req-17fbf6da-e58a-46ca-8e79-1422ece08e38 None None] Registered counter for database keystone Aug 02 09:45:09.285672 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG oslo_db.sqlalchemy.engines [None req-17fbf6da-e58a-46ca-8e79-1422ece08e38 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80153) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 02 09:45:09.291829 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writer thread running {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 02 09:45:09.409112 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fbf6da-e58a-46ca-8e79-1422ece08e38 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:09.413880 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d96bd25-6c7f-4b34-8e42-48d063469421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:09.414539 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d96bd25-6c7f-4b34-8e42-48d063469421 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:09.414825 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d96bd25-6c7f-4b34-8e42-48d063469421 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:09.415102 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d96bd25-6c7f-4b34-8e42-48d063469421 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:09.423146 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d96bd25-6c7f-4b34-8e42-48d063469421 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:09.475277 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: WARNING keystone.common.rbac_enforcer.enforcer [None req-9d96bd25-6c7f-4b34-8e42-48d063469421 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 02 09:45:09.475643 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d96bd25-6c7f-4b34-8e42-48d063469421 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:09.487263 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 2/4] 10.222.0.64 () {58 vars in 1324 bytes} [Tue Aug 2 09:45:09 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 02 09:45:09.493884 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c678105-2c1d-45c8-a0e8-430f9f117410 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:09.508133 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c678105-2c1d-45c8-a0e8-430f9f117410 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:09.512561 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e25816-8f0f-414b-a3e6-bfde77f1e6f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:09.513672 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e25816-8f0f-414b-a3e6-bfde77f1e6f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:09.513955 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e25816-8f0f-414b-a3e6-bfde77f1e6f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:09.514231 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e25816-8f0f-414b-a3e6-bfde77f1e6f6 admin admin] PATH_INFO: `/v3/projects/34e3e6719e2a412f926ff9cdc426bf39` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:09.515699 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e25816-8f0f-414b-a3e6-bfde77f1e6f6 admin admin] RBAC: Authorizing `identity:get_project(project_id=34e3e6719e2a412f926ff9cdc426bf39)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:09.569945 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: WARNING keystone.common.rbac_enforcer.enforcer [None req-48e25816-8f0f-414b-a3e6-bfde77f1e6f6 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 02 09:45:09.570280 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e25816-8f0f-414b-a3e6-bfde77f1e6f6 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:09.575064 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 3/5] 10.222.0.64 () {56 vars in 1207 bytes} [Tue Aug 2 09:45:09 2022] GET /identity/v3/projects/34e3e6719e2a412f926ff9cdc426bf39 => generated 344 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:45:11.543766 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2f69fa-167d-4a15-8044-c1ee79dc0242 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:11.544346 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2f69fa-167d-4a15-8044-c1ee79dc0242 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:11.544346 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2f69fa-167d-4a15-8044-c1ee79dc0242 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:11.548679 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 3/6] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:45:11 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:45:11.555457 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09920841-2935-49ac-bf00-2644c5add248 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:11.555914 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09920841-2935-49ac-bf00-2644c5add248 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:11.556008 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09920841-2935-49ac-bf00-2644c5add248 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:11.576716 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-09920841-2935-49ac-bf00-2644c5add248 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:45:11.578233 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-09920841-2935-49ac-bf00-2644c5add248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:45:11.586026 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 4/7] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:45:11 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 02 09:45:11.608580 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2f69fa-167d-4a15-8044-c1ee79dc0242 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:11.622407 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2f69fa-167d-4a15-8044-c1ee79dc0242 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:11.626662 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539496d0-4669-4df0-84c7-ed5c9c703eb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:11.627300 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539496d0-4669-4df0-84c7-ed5c9c703eb1 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:11.627870 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539496d0-4669-4df0-84c7-ed5c9c703eb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:11.627870 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539496d0-4669-4df0-84c7-ed5c9c703eb1 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:11.636792 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539496d0-4669-4df0-84c7-ed5c9c703eb1 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:11.638602 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539496d0-4669-4df0-84c7-ed5c9c703eb1 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:11.647810 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 4/8] 10.222.0.64 () {58 vars in 1324 bytes} [Tue Aug 2 09:45:11 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 02 09:45:11.653538 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09920841-2935-49ac-bf00-2644c5add248 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:11.663249 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09920841-2935-49ac-bf00-2644c5add248 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:11.667623 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208c42c1-c2c0-4534-9910-b79d3920687a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:11.668270 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208c42c1-c2c0-4534-9910-b79d3920687a admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:11.668575 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208c42c1-c2c0-4534-9910-b79d3920687a admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:11.668853 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208c42c1-c2c0-4534-9910-b79d3920687a admin admin] PATH_INFO: `/v3/users/5c5f9105c00b4d00a6249c76315d5e3e` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:11.670214 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208c42c1-c2c0-4534-9910-b79d3920687a admin admin] RBAC: Authorizing `identity:get_user(user_id=5c5f9105c00b4d00a6249c76315d5e3e)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:11.672031 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208c42c1-c2c0-4534-9910-b79d3920687a admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:11.673624 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 5/9] 10.222.0.64 () {56 vars in 1198 bytes} [Tue Aug 2 09:45:11 2022] GET /identity/v3/users/5c5f9105c00b4d00a6249c76315d5e3e => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:45:13.794018 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b700260-bf92-4e53-a01c-21aecd7f4125 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:13.794565 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b700260-bf92-4e53-a01c-21aecd7f4125 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:13.794565 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b700260-bf92-4e53-a01c-21aecd7f4125 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:13.796091 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 5/10] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:45:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:45:13.800876 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410e1368-974c-4a9b-bc8a-a0771e6b1e21 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:13.801337 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410e1368-974c-4a9b-bc8a-a0771e6b1e21 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:13.801424 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410e1368-974c-4a9b-bc8a-a0771e6b1e21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:13.820400 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-410e1368-974c-4a9b-bc8a-a0771e6b1e21 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:45:13.821609 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-410e1368-974c-4a9b-bc8a-a0771e6b1e21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:45:13.845977 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6ca100-fb6f-4d74-90cc-a3e09f8f6ac2 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:13.846261 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6ca100-fb6f-4d74-90cc-a3e09f8f6ac2 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:13.846538 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6ca100-fb6f-4d74-90cc-a3e09f8f6ac2 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:13.847867 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 6/11] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:45:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 09:45:13.854269 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1432492-987d-49b7-a6c7-a63c48b27706 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:13.854555 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1432492-987d-49b7-a6c7-a63c48b27706 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:13.854828 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1432492-987d-49b7-a6c7-a63c48b27706 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:13.900496 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG passlib.handlers.bcrypt [None req-f1432492-987d-49b7-a6c7-a63c48b27706 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80153) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 02 09:45:13.900943 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG passlib.handlers.bcrypt [None req-f1432492-987d-49b7-a6c7-a63c48b27706 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80153) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 02 09:45:13.904420 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 6/12] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:45:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 09:45:13.926455 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410e1368-974c-4a9b-bc8a-a0771e6b1e21 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:13.947220 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-f1432492-987d-49b7-a6c7-a63c48b27706 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:45:13.948851 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-f1432492-987d-49b7-a6c7-a63c48b27706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:45:13.950412 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:14.032186 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410e1368-974c-4a9b-bc8a-a0771e6b1e21 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:14.050235 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 7/13] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:45:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 09:45:14.067217 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1aeddc-4faf-498d-8a51-d28c2f00b3f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:14.067861 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1aeddc-4faf-498d-8a51-d28c2f00b3f9 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:14.068143 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1aeddc-4faf-498d-8a51-d28c2f00b3f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:14.068457 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1aeddc-4faf-498d-8a51-d28c2f00b3f9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:14.071773 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1432492-987d-49b7-a6c7-a63c48b27706 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:14.080297 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1aeddc-4faf-498d-8a51-d28c2f00b3f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 09:45:14.089160 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-fc1aeddc-4faf-498d-8a51-d28c2f00b3f9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 09:45:14.089160 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:45:14.089160 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:45:14.089160 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:45:14.089160 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:45:14.089160 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:45:14.089160 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:45:14.089160 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:45:14.089160 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:45:14.089160 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:45:14.089160 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:45:14.089160 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:45:14.089160 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 09:45:14.089160 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 09:45:14.089160 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 09:45:14.089160 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 09:45:14.090123 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 09:45:14.090123 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 09:45:14.090123 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 09:45:14.090123 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 09:45:14.090123 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:45:14.090123 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:45:14.090123 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:45:14.090123 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:45:14.090123 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:45:14.090123 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:45:14.090123 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:45:14.090123 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:45:14.090123 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:45:14.090123 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:45:14.090123 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:45:14.090123 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:45:14.090960 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:45:14.090960 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:45:14.090960 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:45:14.090960 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:45:14.090960 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 09:45:14.090960 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 09:45:14.090960 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 09:45:14.090960 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 09:45:14.090960 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 09:45:14.090960 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 09:45:14.090960 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 09:45:14.090960 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:45:14.090960 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 7/14] 10.222.0.64 () {56 vars in 1096 bytes} [Tue Aug 2 09:45:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 09:45:14.096195 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1aeddc-4faf-498d-8a51-d28c2f00b3f9 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:14.177411 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1432492-987d-49b7-a6c7-a63c48b27706 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:14.178362 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:14.179989 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:14.200287 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1aeddc-4faf-498d-8a51-d28c2f00b3f9 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:14.217515 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b5cd59-8b98-4948-baf3-327e4d93cd6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:14.218399 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b5cd59-8b98-4948-baf3-327e4d93cd6b None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:14.218686 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b5cd59-8b98-4948-baf3-327e4d93cd6b None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:14.218963 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b5cd59-8b98-4948-baf3-327e4d93cd6b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:14.235287 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d735801-77d4-4a5a-9f9d-62b31b029eb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:14.235913 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d735801-77d4-4a5a-9f9d-62b31b029eb1 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:14.236201 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d735801-77d4-4a5a-9f9d-62b31b029eb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:14.236508 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d735801-77d4-4a5a-9f9d-62b31b029eb1 None admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:14.237248 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d735801-77d4-4a5a-9f9d-62b31b029eb1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:45:14.237556 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d735801-77d4-4a5a-9f9d-62b31b029eb1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:14.239485 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d735801-77d4-4a5a-9f9d-62b31b029eb1 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:14.252028 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 8/15] 10.222.0.64 () {56 vars in 1099 bytes} [Tue Aug 2 09:45:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:45:14.258349 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d735801-77d4-4a5a-9f9d-62b31b029eb1 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:14.290661 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b5cd59-8b98-4948-baf3-327e4d93cd6b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:14.292209 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b5cd59-8b98-4948-baf3-327e4d93cd6b None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:14.359457 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d735801-77d4-4a5a-9f9d-62b31b029eb1 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:14.393025 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 8/16] 10.222.0.64 () {58 vars in 1282 bytes} [Tue Aug 2 09:45:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 02 09:45:14.399094 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b5cd59-8b98-4948-baf3-327e4d93cd6b None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:14.402304 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:14.501246 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b5cd59-8b98-4948-baf3-327e4d93cd6b None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:14.503387 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed378754-fe8f-4364-a7d8-07a08532a16b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:14.504042 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed378754-fe8f-4364-a7d8-07a08532a16b None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:14.504392 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed378754-fe8f-4364-a7d8-07a08532a16b None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:14.505164 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed378754-fe8f-4364-a7d8-07a08532a16b None admin] PATH_INFO: `/v3/users/5c5f9105c00b4d00a6249c76315d5e3e` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:14.506348 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed378754-fe8f-4364-a7d8-07a08532a16b None admin] RBAC: Authorizing `identity:get_user(user_id=5c5f9105c00b4d00a6249c76315d5e3e)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:14.507806 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed378754-fe8f-4364-a7d8-07a08532a16b None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:14.509443 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 9/17] 10.222.0.64 () {56 vars in 1177 bytes} [Tue Aug 2 09:45:14 2022] GET /identity/v3/users/5c5f9105c00b4d00a6249c76315d5e3e => generated 247 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:45:14.514948 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed378754-fe8f-4364-a7d8-07a08532a16b None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:14.530205 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:14.537723 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e20f6c0-1b31-424c-85cb-c56f89487e55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:14.538594 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e20f6c0-1b31-424c-85cb-c56f89487e55 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:14.538880 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e20f6c0-1b31-424c-85cb-c56f89487e55 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:14.539155 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e20f6c0-1b31-424c-85cb-c56f89487e55 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:14.558002 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e20f6c0-1b31-424c-85cb-c56f89487e55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:14.559486 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e20f6c0-1b31-424c-85cb-c56f89487e55 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:14.585085 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-4e20f6c0-1b31-424c-85cb-c56f89487e55 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:45:14.585085 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:45:14.585085 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:45:14.585085 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:45:14.585085 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:45:14.585085 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:45:14.585085 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:45:14.585085 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:45:14.585085 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:45:14.585085 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:45:14.585085 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:45:14.585085 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:45:14.585085 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 09:45:14.585085 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 09:45:14.585085 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 09:45:14.586087 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 09:45:14.586087 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:45:14.586087 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:45:14.586087 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:45:14.586087 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:45:14.586087 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:45:14.586087 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:45:14.586087 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:45:14.586087 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:45:14.586087 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:45:14.586087 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:45:14.586087 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:45:14.586087 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:45:14.586087 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:45:14.586087 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:45:14.586087 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:45:14.586087 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:45:14.586902 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 09:45:14.586902 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 09:45:14.586902 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:45:14.586902 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:45:14.586902 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 9/18] 10.222.0.64 () {56 vars in 1108 bytes} [Tue Aug 2 09:45:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 09:45:14.592623 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e20f6c0-1b31-424c-85cb-c56f89487e55 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:14.618937 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed378754-fe8f-4364-a7d8-07a08532a16b None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:14.652548 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:14.654541 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff95658-d11d-4971-98f8-7ed8297e5306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:14.655270 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff95658-d11d-4971-98f8-7ed8297e5306 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:14.655558 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff95658-d11d-4971-98f8-7ed8297e5306 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:14.655840 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff95658-d11d-4971-98f8-7ed8297e5306 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:14.657041 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff95658-d11d-4971-98f8-7ed8297e5306 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:14.658484 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff95658-d11d-4971-98f8-7ed8297e5306 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:14.660076 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 10/19] 10.222.0.64 () {56 vars in 1108 bytes} [Tue Aug 2 09:45:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:45:14.665827 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff95658-d11d-4971-98f8-7ed8297e5306 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:14.700704 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e20f6c0-1b31-424c-85cb-c56f89487e55 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:14.737300 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdffc09-fcd4-4a27-bee6-af5ae42b914f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:14.737933 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdffc09-fcd4-4a27-bee6-af5ae42b914f None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:14.738219 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdffc09-fcd4-4a27-bee6-af5ae42b914f None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:14.738494 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdffc09-fcd4-4a27-bee6-af5ae42b914f None admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:14.739783 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdffc09-fcd4-4a27-bee6-af5ae42b914f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:45:14.739783 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdffc09-fcd4-4a27-bee6-af5ae42b914f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:14.740956 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdffc09-fcd4-4a27-bee6-af5ae42b914f None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:14.748400 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:14.759470 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 10/20] 10.222.0.64 () {56 vars in 1109 bytes} [Tue Aug 2 09:45:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:45:14.773192 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff95658-d11d-4971-98f8-7ed8297e5306 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:14.810019 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3b6688-034b-48aa-90b3-9b977a941bb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:14.810731 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3b6688-034b-48aa-90b3-9b977a941bb8 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:14.810939 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3b6688-034b-48aa-90b3-9b977a941bb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:14.811214 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3b6688-034b-48aa-90b3-9b977a941bb8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:14.812035 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3b6688-034b-48aa-90b3-9b977a941bb8 None admin] RBAC: Adding query filter params (role.id=1000cf318fa14915b8091599eaead5de, scope.domain.id=default, user.id=5c5f9105c00b4d00a6249c76315d5e3e) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:45:14.812383 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3b6688-034b-48aa-90b3-9b977a941bb8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:14.813873 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3b6688-034b-48aa-90b3-9b977a941bb8 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:14.822271 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 11/21] 10.222.0.64 () {56 vars in 1322 bytes} [Tue Aug 2 09:45:14 2022] GET /identity/v3/role_assignments?user.id=5c5f9105c00b4d00a6249c76315d5e3e&scope.domain.id=default&role.id=1000cf318fa14915b8091599eaead5de => generated 233 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:45:17.197630 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dd24cf-ff5c-4e60-aa49-03ddf59be2a3 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:17.198107 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dd24cf-ff5c-4e60-aa49-03ddf59be2a3 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:17.198107 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dd24cf-ff5c-4e60-aa49-03ddf59be2a3 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:17.199458 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 11/22] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:45:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:45:17.206353 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8761de-aed3-4026-adfc-dbd6ee85e9c4 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:17.206821 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8761de-aed3-4026-adfc-dbd6ee85e9c4 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:17.207009 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8761de-aed3-4026-adfc-dbd6ee85e9c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:17.231402 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-be8761de-aed3-4026-adfc-dbd6ee85e9c4 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:45:17.232669 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-be8761de-aed3-4026-adfc-dbd6ee85e9c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:45:17.266836 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96535247-768a-45ab-8a94-6503b7dea875 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:17.267131 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96535247-768a-45ab-8a94-6503b7dea875 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:17.267409 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96535247-768a-45ab-8a94-6503b7dea875 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:17.269063 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 12/23] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:45:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 09:45:17.276470 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970fcb4a-2192-45c5-8120-e5f1ffad2628 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:17.276758 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970fcb4a-2192-45c5-8120-e5f1ffad2628 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:17.277042 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970fcb4a-2192-45c5-8120-e5f1ffad2628 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:17.308771 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-970fcb4a-2192-45c5-8120-e5f1ffad2628 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:45:17.309986 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-970fcb4a-2192-45c5-8120-e5f1ffad2628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:45:17.369154 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 12/24] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:45:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 09:45:17.420561 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8761de-aed3-4026-adfc-dbd6ee85e9c4 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:17.455004 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 13/25] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:45:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 09:45:17.456223 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b924c0-2121-4e5c-900b-0cae1b6a5471 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:17.456565 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b924c0-2121-4e5c-900b-0cae1b6a5471 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:17.456839 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b924c0-2121-4e5c-900b-0cae1b6a5471 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:17.462385 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 14/26] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:45:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:45:17.463622 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a42ef68-6545-4f5d-90a5-5f0f88626292 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:17.463912 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a42ef68-6545-4f5d-90a5-5f0f88626292 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:17.464203 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a42ef68-6545-4f5d-90a5-5f0f88626292 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:17.507831 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-0a42ef68-6545-4f5d-90a5-5f0f88626292 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:45:17.509126 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-0a42ef68-6545-4f5d-90a5-5f0f88626292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:45:17.554604 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:17.605594 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:17.618576 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8761de-aed3-4026-adfc-dbd6ee85e9c4 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:17.640148 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 15/27] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:45:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 09:45:17.642043 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a42ef68-6545-4f5d-90a5-5f0f88626292 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:17.708604 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b275a552-586c-4cca-9dc2-ab13e2fbec56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:17.709520 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b275a552-586c-4cca-9dc2-ab13e2fbec56 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:17.709964 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b275a552-586c-4cca-9dc2-ab13e2fbec56 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:17.710386 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b275a552-586c-4cca-9dc2-ab13e2fbec56 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:17.735417 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b275a552-586c-4cca-9dc2-ab13e2fbec56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 09:45:17.742585 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-b275a552-586c-4cca-9dc2-ab13e2fbec56 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 09:45:17.742585 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:45:17.742585 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:45:17.742585 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:45:17.742585 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:45:17.742585 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:45:17.742585 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:45:17.742585 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:45:17.742585 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:45:17.742585 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:45:17.742585 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:45:17.742585 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:45:17.742585 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 09:45:17.742585 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 09:45:17.742585 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 09:45:17.742585 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 09:45:17.743516 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 09:45:17.743516 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 09:45:17.743516 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 09:45:17.743516 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 09:45:17.743516 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:45:17.743516 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:45:17.743516 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:45:17.743516 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:45:17.743516 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:45:17.743516 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:45:17.743516 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:45:17.743516 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:45:17.743516 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:45:17.743516 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:45:17.743516 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:45:17.743516 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:45:17.744292 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:45:17.744292 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:45:17.744292 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:45:17.744292 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:45:17.744292 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 09:45:17.744292 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 09:45:17.744292 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 09:45:17.744292 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 09:45:17.744292 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 09:45:17.744292 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 09:45:17.744292 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 09:45:17.744292 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:45:17.747074 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 13/28] 10.222.0.64 () {56 vars in 1096 bytes} [Tue Aug 2 09:45:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:45:17.748557 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b275a552-586c-4cca-9dc2-ab13e2fbec56 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:17.882612 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a42ef68-6545-4f5d-90a5-5f0f88626292 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:17.896483 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b275a552-586c-4cca-9dc2-ab13e2fbec56 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:17.940998 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49f22b2-b05b-49cd-a418-6499195ae5e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:17.941713 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49f22b2-b05b-49cd-a418-6499195ae5e4 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:17.941996 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49f22b2-b05b-49cd-a418-6499195ae5e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:17.942270 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49f22b2-b05b-49cd-a418-6499195ae5e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:17.946183 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:17.962741 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49f22b2-b05b-49cd-a418-6499195ae5e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:17.964197 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49f22b2-b05b-49cd-a418-6499195ae5e4 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:17.978156 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93eeb24-b794-4c8d-961e-f8a49bb7fc5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:17.978813 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93eeb24-b794-4c8d-961e-f8a49bb7fc5f None admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:17.979146 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93eeb24-b794-4c8d-961e-f8a49bb7fc5f None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:17.979425 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93eeb24-b794-4c8d-961e-f8a49bb7fc5f None admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:17.980198 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93eeb24-b794-4c8d-961e-f8a49bb7fc5f None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:17.981722 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93eeb24-b794-4c8d-961e-f8a49bb7fc5f None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:18.004762 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-a49f22b2-b05b-49cd-a418-6499195ae5e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:45:18.004762 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:45:18.004762 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:45:18.004762 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:45:18.004762 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:45:18.004762 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:45:18.004762 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:45:18.004762 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:45:18.004762 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:45:18.004762 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:45:18.004762 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:45:18.004762 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:45:18.004762 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 09:45:18.004762 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 09:45:18.004762 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 09:45:18.005656 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 09:45:18.005656 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:45:18.005656 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:45:18.005656 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:45:18.005656 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:45:18.005656 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:45:18.005656 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:45:18.005656 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:45:18.005656 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:45:18.005656 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:45:18.005656 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:45:18.005656 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:45:18.005656 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:45:18.005656 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:45:18.005656 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:45:18.005656 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:45:18.005656 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:45:18.006499 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 09:45:18.006499 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 09:45:18.006499 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:45:18.006499 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:45:18.006499 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 14/29] 10.222.0.64 () {56 vars in 1108 bytes} [Tue Aug 2 09:45:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:45:18.010314 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49f22b2-b05b-49cd-a418-6499195ae5e4 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:18.232282 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49f22b2-b05b-49cd-a418-6499195ae5e4 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:18.259331 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 16/30] 10.222.0.64 () {60 vars in 1131 bytes} [Tue Aug 2 09:45:17 2022] POST /identity/v3/roles => generated 219 bytes in 618 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:45:18.261041 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93eeb24-b794-4c8d-961e-f8a49bb7fc5f None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:18.321885 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6a6871-a797-4aae-b8b1-31283ff3a01c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:18.322787 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6a6871-a797-4aae-b8b1-31283ff3a01c None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:18.323408 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6a6871-a797-4aae-b8b1-31283ff3a01c None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:18.323933 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6a6871-a797-4aae-b8b1-31283ff3a01c None admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:18.325095 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6a6871-a797-4aae-b8b1-31283ff3a01c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:45:18.325657 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6a6871-a797-4aae-b8b1-31283ff3a01c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:18.327339 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6a6871-a797-4aae-b8b1-31283ff3a01c None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:18.333693 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:18.351076 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 15/31] 10.222.0.64 () {56 vars in 1099 bytes} [Tue Aug 2 09:45:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:45:18.352918 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edff24dd-5e08-42c7-a7c7-da1b478c6cfe None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:18.356294 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edff24dd-5e08-42c7-a7c7-da1b478c6cfe None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:18.357763 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edff24dd-5e08-42c7-a7c7-da1b478c6cfe None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:18.360357 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 16/32] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:45:18 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:45:18.361557 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edff24dd-5e08-42c7-a7c7-da1b478c6cfe None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:18.429516 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93eeb24-b794-4c8d-961e-f8a49bb7fc5f None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:18.433582 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:18.465103 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa71123-1fb5-4ce3-9876-8122c83307fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:18.465968 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa71123-1fb5-4ce3-9876-8122c83307fa None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:18.466407 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa71123-1fb5-4ce3-9876-8122c83307fa None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:18.466684 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa71123-1fb5-4ce3-9876-8122c83307fa None admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:18.467416 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa71123-1fb5-4ce3-9876-8122c83307fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:45:18.467728 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa71123-1fb5-4ce3-9876-8122c83307fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:18.469243 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa71123-1fb5-4ce3-9876-8122c83307fa None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:18.482153 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 17/33] 10.222.0.64 () {56 vars in 1109 bytes} [Tue Aug 2 09:45:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:45:18.483782 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178d8f0e-171a-4745-923b-f9120e8e70fc None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:18.484201 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178d8f0e-171a-4745-923b-f9120e8e70fc None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:18.484657 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178d8f0e-171a-4745-923b-f9120e8e70fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:18.504047 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edff24dd-5e08-42c7-a7c7-da1b478c6cfe None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:18.509187 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-178d8f0e-171a-4745-923b-f9120e8e70fc None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:45:18.510745 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-178d8f0e-171a-4745-923b-f9120e8e70fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:45:18.543059 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973a39b0-0f1e-425a-af18-ec0ecb04702f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:18.543809 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973a39b0-0f1e-425a-af18-ec0ecb04702f None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:18.544166 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973a39b0-0f1e-425a-af18-ec0ecb04702f None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:18.544500 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973a39b0-0f1e-425a-af18-ec0ecb04702f None admin] PATH_INFO: `/v3/users/5c5f9105c00b4d00a6249c76315d5e3e` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:18.545754 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973a39b0-0f1e-425a-af18-ec0ecb04702f None admin] RBAC: Authorizing `identity:get_user(user_id=5c5f9105c00b4d00a6249c76315d5e3e)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:18.547196 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973a39b0-0f1e-425a-af18-ec0ecb04702f None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:18.548848 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 17/34] 10.222.0.64 () {56 vars in 1177 bytes} [Tue Aug 2 09:45:18 2022] GET /identity/v3/users/5c5f9105c00b4d00a6249c76315d5e3e => generated 247 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:45:18.549975 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31125f3-1fc8-4470-92c9-8df6025a9144 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:18.550261 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31125f3-1fc8-4470-92c9-8df6025a9144 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:18.550540 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31125f3-1fc8-4470-92c9-8df6025a9144 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:18.556525 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 18/35] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:45:18 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:45:18.557310 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31125f3-1fc8-4470-92c9-8df6025a9144 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:18.582582 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 18/36] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:45:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 09:45:18.583434 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178d8f0e-171a-4745-923b-f9120e8e70fc None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:18.611794 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:18.684838 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31125f3-1fc8-4470-92c9-8df6025a9144 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:18.689879 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178d8f0e-171a-4745-923b-f9120e8e70fc None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:18.700766 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:18.720842 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1be985e-53a8-41d1-b698-c87d01fc7cb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:18.721455 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1be985e-53a8-41d1-b698-c87d01fc7cb8 None admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:18.721740 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1be985e-53a8-41d1-b698-c87d01fc7cb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:18.722020 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1be985e-53a8-41d1-b698-c87d01fc7cb8 None admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:18.722700 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1be985e-53a8-41d1-b698-c87d01fc7cb8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:18.724687 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1be985e-53a8-41d1-b698-c87d01fc7cb8 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:18.725772 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc3e57d-e873-4799-aa17-115e03908ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:18.726425 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc3e57d-e873-4799-aa17-115e03908ca8 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:18.726714 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc3e57d-e873-4799-aa17-115e03908ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:18.727255 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc3e57d-e873-4799-aa17-115e03908ca8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:18.728703 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc3e57d-e873-4799-aa17-115e03908ca8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:18.730166 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc3e57d-e873-4799-aa17-115e03908ca8 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:18.731787 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 19/37] 10.222.0.64 () {56 vars in 1108 bytes} [Tue Aug 2 09:45:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:45:18.732983 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543fd312-87a7-4758-b7ba-409ae63f5ede None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:18.733264 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543fd312-87a7-4758-b7ba-409ae63f5ede None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:18.733543 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543fd312-87a7-4758-b7ba-409ae63f5ede None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:18.755476 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-543fd312-87a7-4758-b7ba-409ae63f5ede None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:45:18.757299 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-543fd312-87a7-4758-b7ba-409ae63f5ede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:45:18.831117 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 20/38] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:45:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 09:45:18.831967 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543fd312-87a7-4758-b7ba-409ae63f5ede None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:18.880277 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 19/39] 10.222.0.64 () {60 vars in 1141 bytes} [Tue Aug 2 09:45:18 2022] POST /identity/v3/projects => generated 301 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:45:18.880277 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec9d913-19f1-4f07-a594-ab32217dde21 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:18.880277 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec9d913-19f1-4f07-a594-ab32217dde21 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:18.880277 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec9d913-19f1-4f07-a594-ab32217dde21 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:18.880277 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 20/40] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:45:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:45:18.880277 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec9d913-19f1-4f07-a594-ab32217dde21 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:18.881527 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:18.995517 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543fd312-87a7-4758-b7ba-409ae63f5ede None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:19.016981 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec9d913-19f1-4f07-a594-ab32217dde21 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:19.040623 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d89ddf-3ada-4aa5-b748-e620af9b17bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:19.041763 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d89ddf-3ada-4aa5-b748-e620af9b17bb None admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:19.042065 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d89ddf-3ada-4aa5-b748-e620af9b17bb None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:19.042347 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d89ddf-3ada-4aa5-b748-e620af9b17bb None admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:19.043084 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d89ddf-3ada-4aa5-b748-e620af9b17bb None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:19.044532 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d89ddf-3ada-4aa5-b748-e620af9b17bb None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:19.061382 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f358534-b62b-4c31-9df2-644cdcf944f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:19.061833 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 21/41] 10.222.0.64 () {60 vars in 1131 bytes} [Tue Aug 2 09:45:18 2022] POST /identity/v3/roles => generated 225 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:45:19.062716 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f358534-b62b-4c31-9df2-644cdcf944f8 None admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:19.063223 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f358534-b62b-4c31-9df2-644cdcf944f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:19.063324 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d89ddf-3ada-4aa5-b748-e620af9b17bb None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:19.063790 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f358534-b62b-4c31-9df2-644cdcf944f8 None admin] PATH_INFO: `/v3/domains/default/users/5c5f9105c00b4d00a6249c76315d5e3e/roles/1000cf318fa14915b8091599eaead5de` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:19.066521 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f358534-b62b-4c31-9df2-644cdcf944f8 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=5c5f9105c00b4d00a6249c76315d5e3e, role_id=1000cf318fa14915b8091599eaead5de)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:19.068819 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f358534-b62b-4c31-9df2-644cdcf944f8 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:19.095782 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 21/42] 10.222.0.64 () {58 vars in 1361 bytes} [Tue Aug 2 09:45:18 2022] PUT /identity/v3/domains/default/users/5c5f9105c00b4d00a6249c76315d5e3e/roles/1000cf318fa14915b8091599eaead5de => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:45:19.102945 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa58acf-ba9f-4067-b86d-be28612235df None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:19.102945 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa58acf-ba9f-4067-b86d-be28612235df None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:19.102945 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa58acf-ba9f-4067-b86d-be28612235df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:19.124048 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-bfa58acf-ba9f-4067-b86d-be28612235df None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:45:19.125973 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-bfa58acf-ba9f-4067-b86d-be28612235df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:45:19.161818 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:19.216180 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:19.228623 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 22/43] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:45:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 09:45:19.245396 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d89ddf-3ada-4aa5-b748-e620af9b17bb None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:19.254370 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa58acf-ba9f-4067-b86d-be28612235df None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:19.295043 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1225c507-c9d7-4776-a57c-0f0726d3386b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:19.295637 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1225c507-c9d7-4776-a57c-0f0726d3386b None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:19.295939 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1225c507-c9d7-4776-a57c-0f0726d3386b None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:19.296207 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1225c507-c9d7-4776-a57c-0f0726d3386b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:19.300067 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1225c507-c9d7-4776-a57c-0f0726d3386b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:19.300067 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1225c507-c9d7-4776-a57c-0f0726d3386b None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:19.301617 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 22/44] 10.222.0.64 () {56 vars in 1108 bytes} [Tue Aug 2 09:45:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:45:19.307410 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1225c507-c9d7-4776-a57c-0f0726d3386b None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:19.380555 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa58acf-ba9f-4067-b86d-be28612235df None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:19.417039 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5123c891-a962-4c1d-b560-3fc14a1afe79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:19.417676 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5123c891-a962-4c1d-b560-3fc14a1afe79 None admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:19.417964 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5123c891-a962-4c1d-b560-3fc14a1afe79 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:19.418245 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5123c891-a962-4c1d-b560-3fc14a1afe79 None admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:19.419428 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5123c891-a962-4c1d-b560-3fc14a1afe79 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:19.420874 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5123c891-a962-4c1d-b560-3fc14a1afe79 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:19.432123 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1225c507-c9d7-4776-a57c-0f0726d3386b None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:19.438556 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 23/45] 10.222.0.64 () {60 vars in 1131 bytes} [Tue Aug 2 09:45:19 2022] POST /identity/v3/roles => generated 223 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 09:45:19.471205 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523fd48a-ce46-4bc7-82ce-d73cda49839b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:19.471880 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523fd48a-ce46-4bc7-82ce-d73cda49839b None admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:19.472150 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523fd48a-ce46-4bc7-82ce-d73cda49839b None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:19.472487 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523fd48a-ce46-4bc7-82ce-d73cda49839b None admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:19.473833 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523fd48a-ce46-4bc7-82ce-d73cda49839b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:19.475369 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523fd48a-ce46-4bc7-82ce-d73cda49839b None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:19.512300 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 23/46] 10.222.0.64 () {60 vars in 1141 bytes} [Tue Aug 2 09:45:19 2022] POST /identity/v3/projects => generated 312 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 09:45:21.199368 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aadcc9-a580-4373-a553-062e0ce786bc None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:21.199868 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aadcc9-a580-4373-a553-062e0ce786bc None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:21.199868 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aadcc9-a580-4373-a553-062e0ce786bc None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:21.201249 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 24/47] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:45:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:45:21.206699 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2e4b66-f224-47bb-b9d9-f06f3d800ec4 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:21.207683 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2e4b66-f224-47bb-b9d9-f06f3d800ec4 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:21.207775 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2e4b66-f224-47bb-b9d9-f06f3d800ec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:21.215841 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:21.229061 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-df2e4b66-f224-47bb-b9d9-f06f3d800ec4 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:45:21.230264 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-df2e4b66-f224-47bb-b9d9-f06f3d800ec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:45:21.297880 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 24/48] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:45:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 09:45:21.320716 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67aadcc9-a580-4373-a553-062e0ce786bc None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:21.341709 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:21.438760 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67aadcc9-a580-4373-a553-062e0ce786bc None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:21.476019 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869772eb-3709-4c73-9ecd-805c3aad537b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:21.476834 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869772eb-3709-4c73-9ecd-805c3aad537b None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:21.477416 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869772eb-3709-4c73-9ecd-805c3aad537b None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:21.477808 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869772eb-3709-4c73-9ecd-805c3aad537b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:21.489359 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869772eb-3709-4c73-9ecd-805c3aad537b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 09:45:21.494315 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-869772eb-3709-4c73-9ecd-805c3aad537b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 09:45:21.494315 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:45:21.494315 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:45:21.494315 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:45:21.494315 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:45:21.494315 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:45:21.494315 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:45:21.494315 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:45:21.494315 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:45:21.494315 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:45:21.494315 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:45:21.494315 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:45:21.494315 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 09:45:21.494315 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 09:45:21.494315 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 09:45:21.494315 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 09:45:21.495265 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 09:45:21.495265 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 09:45:21.495265 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 09:45:21.495265 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 09:45:21.495265 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:45:21.495265 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:45:21.495265 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:45:21.495265 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:45:21.495265 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:45:21.495265 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:45:21.495265 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:45:21.495265 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:45:21.495265 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:45:21.495265 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:45:21.495265 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:45:21.495265 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:45:21.496119 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:45:21.496119 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:45:21.496119 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:45:21.496119 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:45:21.496119 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 09:45:21.496119 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 09:45:21.496119 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 09:45:21.496119 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 09:45:21.496119 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 09:45:21.496119 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 09:45:21.496119 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 09:45:21.496119 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:45:21.496119 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 25/49] 10.222.0.64 () {56 vars in 1096 bytes} [Tue Aug 2 09:45:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:45:21.501471 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2e4b66-f224-47bb-b9d9-f06f3d800ec4 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:21.558361 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a906fa6-4400-4757-b9f5-50465aee39d1 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:21.558653 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a906fa6-4400-4757-b9f5-50465aee39d1 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:21.558936 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a906fa6-4400-4757-b9f5-50465aee39d1 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:21.560401 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 26/50] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:45:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 09:45:21.565710 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a968678e-321a-4266-90d8-fa5b2f9938b0 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:21.565997 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a968678e-321a-4266-90d8-fa5b2f9938b0 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:21.566278 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a968678e-321a-4266-90d8-fa5b2f9938b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:21.585002 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-a968678e-321a-4266-90d8-fa5b2f9938b0 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:45:21.586211 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-a968678e-321a-4266-90d8-fa5b2f9938b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:45:21.615548 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2e4b66-f224-47bb-b9d9-f06f3d800ec4 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:21.651984 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d6b870-777d-41aa-b83a-8fd10e7300b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:21.652653 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d6b870-777d-41aa-b83a-8fd10e7300b0 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:21.652945 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d6b870-777d-41aa-b83a-8fd10e7300b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:21.653224 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d6b870-777d-41aa-b83a-8fd10e7300b0 None admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:21.653959 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d6b870-777d-41aa-b83a-8fd10e7300b0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:45:21.654272 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d6b870-777d-41aa-b83a-8fd10e7300b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:21.655689 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d6b870-777d-41aa-b83a-8fd10e7300b0 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:21.657431 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:21.662835 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 27/51] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:45:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 09:45:21.676963 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 25/52] 10.222.0.64 () {56 vars in 1099 bytes} [Tue Aug 2 09:45:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:45:21.682406 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a968678e-321a-4266-90d8-fa5b2f9938b0 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:21.685421 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d6b870-777d-41aa-b83a-8fd10e7300b0 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:21.693394 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:21.793346 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d6b870-777d-41aa-b83a-8fd10e7300b0 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:21.799535 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a968678e-321a-4266-90d8-fa5b2f9938b0 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:21.829495 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4706ff31-4d5a-414e-9636-13fbd4a5a77c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:21.830151 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4706ff31-4d5a-414e-9636-13fbd4a5a77c None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:21.830438 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4706ff31-4d5a-414e-9636-13fbd4a5a77c None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:21.830726 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4706ff31-4d5a-414e-9636-13fbd4a5a77c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:21.831906 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4706ff31-4d5a-414e-9636-13fbd4a5a77c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:21.833448 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4706ff31-4d5a-414e-9636-13fbd4a5a77c None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:21.835237 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 26/53] 10.222.0.64 () {56 vars in 1108 bytes} [Tue Aug 2 09:45:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:45:21.835607 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a7d862-d8f8-4ade-a4fa-8d50e1baf44c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:21.836379 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a7d862-d8f8-4ade-a4fa-8d50e1baf44c None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:21.836676 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a7d862-d8f8-4ade-a4fa-8d50e1baf44c None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:21.836962 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a7d862-d8f8-4ade-a4fa-8d50e1baf44c None admin] PATH_INFO: `/v3/users/5c5f9105c00b4d00a6249c76315d5e3e` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:21.838421 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a7d862-d8f8-4ade-a4fa-8d50e1baf44c None admin] RBAC: Authorizing `identity:get_user(user_id=5c5f9105c00b4d00a6249c76315d5e3e)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:21.840021 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a7d862-d8f8-4ade-a4fa-8d50e1baf44c None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:21.843238 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4706ff31-4d5a-414e-9636-13fbd4a5a77c None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:21.843376 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 28/54] 10.222.0.64 () {56 vars in 1177 bytes} [Tue Aug 2 09:45:21 2022] GET /identity/v3/users/5c5f9105c00b4d00a6249c76315d5e3e => generated 247 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:45:21.847370 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a7d862-d8f8-4ade-a4fa-8d50e1baf44c None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:21.912257 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:21.929113 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:21.951799 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4706ff31-4d5a-414e-9636-13fbd4a5a77c None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:21.953397 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a7d862-d8f8-4ade-a4fa-8d50e1baf44c None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:21.985895 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed2245d-6b26-4e06-a9e8-e23af5d14673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:21.986522 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed2245d-6b26-4e06-a9e8-e23af5d14673 None admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:21.986815 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed2245d-6b26-4e06-a9e8-e23af5d14673 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:21.987076 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8fc3ea-219f-46e3-91bd-479620900259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:21.987182 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed2245d-6b26-4e06-a9e8-e23af5d14673 None admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:21.987715 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8fc3ea-219f-46e3-91bd-479620900259 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:21.987803 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed2245d-6b26-4e06-a9e8-e23af5d14673 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:21.987999 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8fc3ea-219f-46e3-91bd-479620900259 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:21.988992 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8fc3ea-219f-46e3-91bd-479620900259 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:21.989743 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed2245d-6b26-4e06-a9e8-e23af5d14673 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:21.990162 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8fc3ea-219f-46e3-91bd-479620900259 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:21.991608 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8fc3ea-219f-46e3-91bd-479620900259 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:21.993220 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 29/55] 10.222.0.64 () {56 vars in 1108 bytes} [Tue Aug 2 09:45:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:45:21.999017 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8fc3ea-219f-46e3-91bd-479620900259 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:22.017604 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 27/56] 10.222.0.64 () {60 vars in 1140 bytes} [Tue Aug 2 09:45:21 2022] POST /identity/v3/projects => generated 298 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 09:45:22.109516 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8fc3ea-219f-46e3-91bd-479620900259 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:22.146276 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45dc89f-2296-4d8d-a39d-07413b0a5187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:22.146915 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45dc89f-2296-4d8d-a39d-07413b0a5187 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:22.147202 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45dc89f-2296-4d8d-a39d-07413b0a5187 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:22.147486 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45dc89f-2296-4d8d-a39d-07413b0a5187 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:22.148331 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45dc89f-2296-4d8d-a39d-07413b0a5187 None admin] RBAC: Adding query filter params (role.id=1000cf318fa14915b8091599eaead5de, scope.domain.id=default, user.id=5c5f9105c00b4d00a6249c76315d5e3e) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:45:22.148648 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45dc89f-2296-4d8d-a39d-07413b0a5187 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:22.150139 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45dc89f-2296-4d8d-a39d-07413b0a5187 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:22.156792 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 30/57] 10.222.0.64 () {56 vars in 1322 bytes} [Tue Aug 2 09:45:21 2022] GET /identity/v3/role_assignments?user.id=5c5f9105c00b4d00a6249c76315d5e3e&scope.domain.id=default&role.id=1000cf318fa14915b8091599eaead5de => generated 530 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:45:24.047865 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6588747c-f0e9-4145-bf19-9de327574ee8 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:24.048446 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6588747c-f0e9-4145-bf19-9de327574ee8 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:24.048446 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6588747c-f0e9-4145-bf19-9de327574ee8 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:24.049805 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 28/58] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:45:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:45:24.055457 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47baf653-0bcb-4d87-ac25-99fb298b61e9 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:24.055900 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47baf653-0bcb-4d87-ac25-99fb298b61e9 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:24.056068 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47baf653-0bcb-4d87-ac25-99fb298b61e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:24.074336 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-47baf653-0bcb-4d87-ac25-99fb298b61e9 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:45:24.075535 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-47baf653-0bcb-4d87-ac25-99fb298b61e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:45:24.077967 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:24.149677 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 31/59] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:45:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 09:45:24.173670 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6588747c-f0e9-4145-bf19-9de327574ee8 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:24.278795 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6588747c-f0e9-4145-bf19-9de327574ee8 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:24.295445 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:24.313282 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8582342b-e45d-4e68-92be-521ffa8cc3e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:24.313983 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8582342b-e45d-4e68-92be-521ffa8cc3e6 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:24.314269 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8582342b-e45d-4e68-92be-521ffa8cc3e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:24.314545 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8582342b-e45d-4e68-92be-521ffa8cc3e6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:24.315696 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8582342b-e45d-4e68-92be-521ffa8cc3e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:24.317265 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8582342b-e45d-4e68-92be-521ffa8cc3e6 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:24.318854 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 29/60] 10.222.0.64 () {56 vars in 1108 bytes} [Tue Aug 2 09:45:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:45:24.325282 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47baf653-0bcb-4d87-ac25-99fb298b61e9 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:24.425685 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47baf653-0bcb-4d87-ac25-99fb298b61e9 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:24.460066 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b71b270-b669-4ed9-8d70-4bf64cef58b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:24.460726 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b71b270-b669-4ed9-8d70-4bf64cef58b0 None admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:24.461018 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b71b270-b669-4ed9-8d70-4bf64cef58b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:24.461302 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b71b270-b669-4ed9-8d70-4bf64cef58b0 None admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:24.461999 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b71b270-b669-4ed9-8d70-4bf64cef58b0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:24.463475 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b71b270-b669-4ed9-8d70-4bf64cef58b0 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:24.477975 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:24.492980 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 32/61] 10.222.0.64 () {60 vars in 1132 bytes} [Tue Aug 2 09:45:24 2022] POST /identity/v3/users => generated 275 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:45:27.624965 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bad3a2a-fcbe-426d-8e27-a156648eab07 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:27.625820 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bad3a2a-fcbe-426d-8e27-a156648eab07 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:27.626248 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bad3a2a-fcbe-426d-8e27-a156648eab07 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:27.628008 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 30/62] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:45:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:45:27.633717 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66802e4c-2947-4475-98d0-ee2b57519a6d None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:27.634563 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66802e4c-2947-4475-98d0-ee2b57519a6d None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:27.635080 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66802e4c-2947-4475-98d0-ee2b57519a6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:27.653817 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-66802e4c-2947-4475-98d0-ee2b57519a6d None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:45:27.655375 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-66802e4c-2947-4475-98d0-ee2b57519a6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:45:27.729992 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 33/63] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:45:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 09:45:27.770872 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bad3a2a-fcbe-426d-8e27-a156648eab07 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:27.902002 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bad3a2a-fcbe-426d-8e27-a156648eab07 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:27.929967 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89faefe0-9e54-4f81-bdfd-86b651a7678c None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:27.930596 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89faefe0-9e54-4f81-bdfd-86b651a7678c None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:27.931080 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89faefe0-9e54-4f81-bdfd-86b651a7678c None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:27.933014 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 34/64] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:45:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:45:27.938431 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004e995e-f64d-4a4b-8f1a-99519cb7fa6d None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:27.938962 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004e995e-f64d-4a4b-8f1a-99519cb7fa6d None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:27.939484 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004e995e-f64d-4a4b-8f1a-99519cb7fa6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:27.947859 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc6bc0a-2e4d-4eec-b6b0-9597edacf9c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:27.948555 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc6bc0a-2e4d-4eec-b6b0-9597edacf9c7 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:27.949721 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc6bc0a-2e4d-4eec-b6b0-9597edacf9c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:27.949721 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc6bc0a-2e4d-4eec-b6b0-9597edacf9c7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:27.960826 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-004e995e-f64d-4a4b-8f1a-99519cb7fa6d None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:45:27.962594 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-004e995e-f64d-4a4b-8f1a-99519cb7fa6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:45:27.966905 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc6bc0a-2e4d-4eec-b6b0-9597edacf9c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 09:45:27.985669 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-fbc6bc0a-2e4d-4eec-b6b0-9597edacf9c7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 02 09:45:27.985669 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:45:27.985669 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:45:27.985669 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:45:27.985669 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:45:27.985669 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:45:27.985669 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:45:27.985669 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:45:27.985669 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:45:27.985669 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:45:27.985669 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:45:27.985669 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:45:27.985669 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 09:45:27.985669 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 09:45:27.985669 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 09:45:27.985669 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 09:45:27.986631 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 09:45:27.986631 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 09:45:27.986631 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 09:45:27.986631 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 09:45:27.986631 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:45:27.986631 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:45:27.986631 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:45:27.986631 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:45:27.986631 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:45:27.986631 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:45:27.986631 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:45:27.986631 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:45:27.986631 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:45:27.986631 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:45:27.986631 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:45:27.986631 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:45:27.987429 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:45:27.987429 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:45:27.987429 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:45:27.987429 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:45:27.987429 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 09:45:27.987429 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 09:45:27.987429 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 09:45:27.987429 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 09:45:27.987429 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 09:45:27.987429 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 09:45:27.987429 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 02 09:45:27.987429 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:45:27.987429 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 31/65] 10.222.0.64 () {56 vars in 1099 bytes} [Tue Aug 2 09:45:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:45:27.993883 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc6bc0a-2e4d-4eec-b6b0-9597edacf9c7 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:28.040542 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 35/66] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:45:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 09:45:28.052378 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b81175-6a44-4472-b5b8-9e26ba2d4397 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:28.052601 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b81175-6a44-4472-b5b8-9e26ba2d4397 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:28.052969 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b81175-6a44-4472-b5b8-9e26ba2d4397 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:28.054409 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 36/67] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:45:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 09:45:28.060702 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b1c5f3-598a-4b8a-9342-5601a205bf18 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:28.060997 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b1c5f3-598a-4b8a-9342-5601a205bf18 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:28.061466 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b1c5f3-598a-4b8a-9342-5601a205bf18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:28.090224 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-26b1c5f3-598a-4b8a-9342-5601a205bf18 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:45:28.091655 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-26b1c5f3-598a-4b8a-9342-5601a205bf18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:45:28.092652 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:28.136865 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc6bc0a-2e4d-4eec-b6b0-9597edacf9c7 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:28.161004 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:28.168977 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 37/68] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:45:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 09:45:28.173795 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b1c5f3-598a-4b8a-9342-5601a205bf18 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:28.181730 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa6e0bf-ee21-47e4-acfd-add3ce362c6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:28.182628 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa6e0bf-ee21-47e4-acfd-add3ce362c6a None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:28.183084 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa6e0bf-ee21-47e4-acfd-add3ce362c6a None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:28.183876 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa6e0bf-ee21-47e4-acfd-add3ce362c6a None admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:28.184810 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa6e0bf-ee21-47e4-acfd-add3ce362c6a None admin] RBAC: Adding query filter params (name=member) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:45:28.185290 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa6e0bf-ee21-47e4-acfd-add3ce362c6a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:28.186888 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa6e0bf-ee21-47e4-acfd-add3ce362c6a None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:28.197350 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 32/69] 10.222.0.64 () {56 vars in 1101 bytes} [Tue Aug 2 09:45:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:45:28.198894 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcffa96-bc2f-409c-a808-f5682fd5ac2a None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:28.199351 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcffa96-bc2f-409c-a808-f5682fd5ac2a None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:28.199907 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcffa96-bc2f-409c-a808-f5682fd5ac2a None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:28.201211 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 33/70] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:45:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:45:28.205847 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcffa96-bc2f-409c-a808-f5682fd5ac2a None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:28.284516 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b1c5f3-598a-4b8a-9342-5601a205bf18 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:28.318575 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcffa96-bc2f-409c-a808-f5682fd5ac2a None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:28.325934 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728879e5-2a39-48ba-8c37-707e11d74299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:28.326333 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:28.326590 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728879e5-2a39-48ba-8c37-707e11d74299 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:28.326885 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728879e5-2a39-48ba-8c37-707e11d74299 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:28.327161 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728879e5-2a39-48ba-8c37-707e11d74299 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:28.339292 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728879e5-2a39-48ba-8c37-707e11d74299 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 09:45:28.344317 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-728879e5-2a39-48ba-8c37-707e11d74299 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 09:45:28.344317 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:45:28.344317 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:45:28.344317 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:45:28.344317 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:45:28.344317 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:45:28.344317 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:45:28.344317 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:45:28.344317 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:45:28.344317 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:45:28.344317 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:45:28.344317 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:45:28.344317 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 09:45:28.344317 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 09:45:28.344317 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 09:45:28.344317 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 09:45:28.345370 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 09:45:28.345370 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 09:45:28.345370 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 09:45:28.345370 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 09:45:28.345370 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:45:28.345370 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:45:28.345370 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:45:28.345370 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:45:28.345370 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:45:28.345370 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:45:28.345370 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:45:28.345370 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:45:28.345370 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:45:28.345370 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:45:28.345370 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:45:28.345370 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:45:28.346159 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:45:28.346159 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:45:28.346159 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:45:28.346159 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:45:28.346159 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 09:45:28.346159 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 09:45:28.346159 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 09:45:28.346159 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 09:45:28.346159 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 09:45:28.346159 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 09:45:28.346159 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 09:45:28.346159 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:45:28.346159 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 38/71] 10.222.0.64 () {56 vars in 1114 bytes} [Tue Aug 2 09:45:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:45:28.346874 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728879e5-2a39-48ba-8c37-707e11d74299 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:28.366038 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4493d7-a508-4c61-bf2d-2eedf0bbb80a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:28.366804 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4493d7-a508-4c61-bf2d-2eedf0bbb80a None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:28.367091 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4493d7-a508-4c61-bf2d-2eedf0bbb80a None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:28.367374 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4493d7-a508-4c61-bf2d-2eedf0bbb80a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:28.380330 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4493d7-a508-4c61-bf2d-2eedf0bbb80a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 09:45:28.385358 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-4c4493d7-a508-4c61-bf2d-2eedf0bbb80a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 09:45:28.385358 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:45:28.385358 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:45:28.385358 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:45:28.385358 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:45:28.385358 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:45:28.385358 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:45:28.385358 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:45:28.385358 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:45:28.385358 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:45:28.385358 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:45:28.385358 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:45:28.385358 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 09:45:28.385358 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 09:45:28.385358 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 09:45:28.385358 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 09:45:28.386416 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 09:45:28.386416 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 09:45:28.386416 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 09:45:28.386416 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 09:45:28.386416 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:45:28.386416 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:45:28.386416 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:45:28.386416 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:45:28.386416 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:45:28.386416 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:45:28.386416 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:45:28.386416 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:45:28.386416 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:45:28.386416 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:45:28.386416 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:45:28.386416 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:45:28.387390 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:45:28.387390 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:45:28.387390 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:45:28.387390 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:45:28.387390 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 09:45:28.387390 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 09:45:28.387390 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 09:45:28.387390 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 09:45:28.387390 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 09:45:28.387390 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 09:45:28.387390 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 09:45:28.387390 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:45:28.387390 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 34/72] 10.222.0.64 () {56 vars in 1096 bytes} [Tue Aug 2 09:45:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:45:28.389671 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83926deb-c71e-472d-92e5-87105c7116cd None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:28.389964 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83926deb-c71e-472d-92e5-87105c7116cd None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:28.390248 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83926deb-c71e-472d-92e5-87105c7116cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:28.401137 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:28.413372 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-83926deb-c71e-472d-92e5-87105c7116cd None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:45:28.414491 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-83926deb-c71e-472d-92e5-87105c7116cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:45:28.461654 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728879e5-2a39-48ba-8c37-707e11d74299 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:28.489280 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 35/73] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:45:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 09:45:28.490141 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83926deb-c71e-472d-92e5-87105c7116cd None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:28.499753 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730f5216-2cbc-4e39-a519-02c9eeb13765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:28.500812 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730f5216-2cbc-4e39-a519-02c9eeb13765 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:28.501113 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730f5216-2cbc-4e39-a519-02c9eeb13765 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:28.501398 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730f5216-2cbc-4e39-a519-02c9eeb13765 None admin] PATH_INFO: `/v3/users/6b35b3486c4b4db6a107718cf2807616` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:28.517653 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730f5216-2cbc-4e39-a519-02c9eeb13765 None admin] RBAC: Authorizing `identity:get_user(user_id=6b35b3486c4b4db6a107718cf2807616)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:28.519169 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730f5216-2cbc-4e39-a519-02c9eeb13765 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:28.521515 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 39/74] 10.222.0.64 () {56 vars in 1177 bytes} [Tue Aug 2 09:45:28 2022] GET /identity/v3/users/6b35b3486c4b4db6a107718cf2807616 => generated 275 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:45:28.522320 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730f5216-2cbc-4e39-a519-02c9eeb13765 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:28.585765 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:28.592297 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83926deb-c71e-472d-92e5-87105c7116cd None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:28.628032 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008e6d6c-25d0-47cb-9561-ca161f11dc1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:28.628796 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008e6d6c-25d0-47cb-9561-ca161f11dc1a None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:28.629076 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008e6d6c-25d0-47cb-9561-ca161f11dc1a None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:28.629362 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008e6d6c-25d0-47cb-9561-ca161f11dc1a None admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:28.630098 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008e6d6c-25d0-47cb-9561-ca161f11dc1a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:45:28.630414 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008e6d6c-25d0-47cb-9561-ca161f11dc1a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:28.631805 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008e6d6c-25d0-47cb-9561-ca161f11dc1a None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:28.632529 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730f5216-2cbc-4e39-a519-02c9eeb13765 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:28.634111 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:28.641139 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 36/75] 10.222.0.64 () {56 vars in 1111 bytes} [Tue Aug 2 09:45:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:45:28.642558 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31b1b29-7a75-4522-b2ef-9945441ad4eb None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:28.642857 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31b1b29-7a75-4522-b2ef-9945441ad4eb None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:28.643137 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31b1b29-7a75-4522-b2ef-9945441ad4eb None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:28.646459 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 37/76] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:45:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:45:28.646459 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31b1b29-7a75-4522-b2ef-9945441ad4eb None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:28.674205 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e8a937-ceb7-4b6d-9a2e-4a4112b8c2eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:28.674756 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e8a937-ceb7-4b6d-9a2e-4a4112b8c2eb None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:28.675046 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e8a937-ceb7-4b6d-9a2e-4a4112b8c2eb None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:28.675327 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e8a937-ceb7-4b6d-9a2e-4a4112b8c2eb None admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:28.676112 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e8a937-ceb7-4b6d-9a2e-4a4112b8c2eb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:45:28.676459 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e8a937-ceb7-4b6d-9a2e-4a4112b8c2eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:28.677865 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e8a937-ceb7-4b6d-9a2e-4a4112b8c2eb None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:28.686228 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 40/77] 10.222.0.64 () {56 vars in 1099 bytes} [Tue Aug 2 09:45:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:45:28.687079 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e8a937-ceb7-4b6d-9a2e-4a4112b8c2eb None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:28.752819 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31b1b29-7a75-4522-b2ef-9945441ad4eb None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:28.787847 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be884832-f265-4795-93cb-01814274cc23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:28.788522 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be884832-f265-4795-93cb-01814274cc23 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:28.788810 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be884832-f265-4795-93cb-01814274cc23 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:28.789092 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be884832-f265-4795-93cb-01814274cc23 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:28.789929 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e8a937-ceb7-4b6d-9a2e-4a4112b8c2eb None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:28.798976 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be884832-f265-4795-93cb-01814274cc23 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 09:45:28.804304 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-be884832-f265-4795-93cb-01814274cc23 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 02 09:45:28.804304 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:45:28.804304 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:45:28.804304 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:45:28.804304 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:45:28.804304 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:45:28.804304 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:45:28.804304 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:45:28.804304 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:45:28.804304 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:45:28.804304 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:45:28.804304 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:45:28.804304 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 09:45:28.804304 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 09:45:28.804304 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 09:45:28.804304 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 09:45:28.805153 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 09:45:28.805153 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 09:45:28.805153 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 09:45:28.805153 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 09:45:28.805153 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:45:28.805153 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:45:28.805153 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:45:28.805153 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:45:28.805153 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:45:28.805153 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:45:28.805153 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:45:28.805153 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:45:28.805153 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:45:28.805153 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:45:28.805153 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:45:28.805153 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:45:28.805938 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:45:28.805938 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:45:28.805938 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:45:28.805938 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:45:28.805938 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 09:45:28.805938 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 09:45:28.805938 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 09:45:28.805938 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 09:45:28.805938 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 09:45:28.805938 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 09:45:28.805938 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 02 09:45:28.805938 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:45:28.805938 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 38/78] 10.222.0.64 () {56 vars in 1099 bytes} [Tue Aug 2 09:45:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:45:28.806570 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be884832-f265-4795-93cb-01814274cc23 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:28.824623 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:28.825263 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f427abe-d76c-4d22-b2b6-43fed583a622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:28.825891 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f427abe-d76c-4d22-b2b6-43fed583a622 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:28.826182 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f427abe-d76c-4d22-b2b6-43fed583a622 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:28.826484 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f427abe-d76c-4d22-b2b6-43fed583a622 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:28.858416 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:28.899144 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f427abe-d76c-4d22-b2b6-43fed583a622 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:28.900707 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f427abe-d76c-4d22-b2b6-43fed583a622 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:28.909986 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be884832-f265-4795-93cb-01814274cc23 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:28.943721 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3a6376-c35d-4b49-87aa-254942d37b9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:28.944327 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3a6376-c35d-4b49-87aa-254942d37b9c None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:28.944614 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3a6376-c35d-4b49-87aa-254942d37b9c None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:28.944887 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3a6376-c35d-4b49-87aa-254942d37b9c None admin] PATH_INFO: `/v3/users/6b35b3486c4b4db6a107718cf2807616` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:28.946281 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3a6376-c35d-4b49-87aa-254942d37b9c None admin] RBAC: Authorizing `identity:get_user(user_id=6b35b3486c4b4db6a107718cf2807616)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:28.947746 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3a6376-c35d-4b49-87aa-254942d37b9c None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:28.949367 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 39/79] 10.222.0.64 () {56 vars in 1177 bytes} [Tue Aug 2 09:45:28 2022] GET /identity/v3/users/6b35b3486c4b4db6a107718cf2807616 => generated 275 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:45:28.950471 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c75ffda-e861-4903-9063-abbfa95c94bf None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:28.950757 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c75ffda-e861-4903-9063-abbfa95c94bf None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:28.951036 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c75ffda-e861-4903-9063-abbfa95c94bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:28.969525 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-7c75ffda-e861-4903-9063-abbfa95c94bf None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:45:28.970717 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-7c75ffda-e861-4903-9063-abbfa95c94bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:45:29.000589 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 41/80] 10.222.0.64 () {58 vars in 1282 bytes} [Tue Aug 2 09:45:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 09:45:29.001432 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f427abe-d76c-4d22-b2b6-43fed583a622 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:29.035496 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 40/81] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:45:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 09:45:29.036191 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c75ffda-e861-4903-9063-abbfa95c94bf None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:29.064674 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:29.065419 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:29.103963 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f427abe-d76c-4d22-b2b6-43fed583a622 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:29.138052 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7c2048-2c87-47a1-8190-0d8f6dd087ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:29.138691 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7c2048-2c87-47a1-8190-0d8f6dd087ae None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:29.138976 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7c2048-2c87-47a1-8190-0d8f6dd087ae None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:29.139256 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7c2048-2c87-47a1-8190-0d8f6dd087ae None admin] PATH_INFO: `/v3/users/5c5f9105c00b4d00a6249c76315d5e3e` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:29.140594 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7c2048-2c87-47a1-8190-0d8f6dd087ae None admin] RBAC: Authorizing `identity:get_user(user_id=5c5f9105c00b4d00a6249c76315d5e3e)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:29.142031 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7c2048-2c87-47a1-8190-0d8f6dd087ae None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:29.143633 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 42/82] 10.222.0.64 () {56 vars in 1177 bytes} [Tue Aug 2 09:45:29 2022] GET /identity/v3/users/5c5f9105c00b4d00a6249c76315d5e3e => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:45:29.145631 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7c2048-2c87-47a1-8190-0d8f6dd087ae None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:29.147081 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c75ffda-e861-4903-9063-abbfa95c94bf None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:29.184051 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74fcc8e-bc25-4f18-b109-78fef886fd2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:29.184710 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74fcc8e-bc25-4f18-b109-78fef886fd2b None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:29.185000 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74fcc8e-bc25-4f18-b109-78fef886fd2b None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:29.185277 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74fcc8e-bc25-4f18-b109-78fef886fd2b None admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:29.186011 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74fcc8e-bc25-4f18-b109-78fef886fd2b None admin] RBAC: Adding query filter params (name=member) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:45:29.186322 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74fcc8e-bc25-4f18-b109-78fef886fd2b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:29.187731 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74fcc8e-bc25-4f18-b109-78fef886fd2b None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:29.196719 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 41/83] 10.222.0.64 () {56 vars in 1101 bytes} [Tue Aug 2 09:45:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:45:29.197479 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74fcc8e-bc25-4f18-b109-78fef886fd2b None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:29.255686 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7c2048-2c87-47a1-8190-0d8f6dd087ae None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:29.282200 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:29.287595 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:29.292934 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3207d50c-1c8b-46d0-91af-e7859cddfe5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:29.293571 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3207d50c-1c8b-46d0-91af-e7859cddfe5e None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:29.293847 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3207d50c-1c8b-46d0-91af-e7859cddfe5e None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:29.294121 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3207d50c-1c8b-46d0-91af-e7859cddfe5e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:29.305478 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74fcc8e-bc25-4f18-b109-78fef886fd2b None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:29.363343 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5990d056-3997-49dc-8e97-64b5b5f82747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:29.364456 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5990d056-3997-49dc-8e97-64b5b5f82747 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:29.364955 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5990d056-3997-49dc-8e97-64b5b5f82747 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:29.365440 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5990d056-3997-49dc-8e97-64b5b5f82747 None admin] PATH_INFO: `/v3/projects/ae78e3e21c0448c0b412e161cc374652` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:29.366341 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3207d50c-1c8b-46d0-91af-e7859cddfe5e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:29.367418 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5990d056-3997-49dc-8e97-64b5b5f82747 None admin] RBAC: Authorizing `identity:get_project(project_id=ae78e3e21c0448c0b412e161cc374652)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:29.367837 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3207d50c-1c8b-46d0-91af-e7859cddfe5e None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:29.370080 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5990d056-3997-49dc-8e97-64b5b5f82747 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:29.378010 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 42/84] 10.222.0.64 () {56 vars in 1186 bytes} [Tue Aug 2 09:45:29 2022] GET /identity/v3/projects/ae78e3e21c0448c0b412e161cc374652 => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:45:29.378010 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5990d056-3997-49dc-8e97-64b5b5f82747 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:29.470751 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 43/85] 10.222.0.64 () {58 vars in 1282 bytes} [Tue Aug 2 09:45:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 09:45:29.471904 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3207d50c-1c8b-46d0-91af-e7859cddfe5e None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:29.483394 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5990d056-3997-49dc-8e97-64b5b5f82747 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:29.503176 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:29.517014 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c96299-7798-4ca8-ba53-e9e1d35e75db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:29.517654 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c96299-7798-4ca8-ba53-e9e1d35e75db None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:29.517938 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c96299-7798-4ca8-ba53-e9e1d35e75db None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:29.518214 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c96299-7798-4ca8-ba53-e9e1d35e75db None admin] PATH_INFO: `/v3/domains/default` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:29.519947 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c96299-7798-4ca8-ba53-e9e1d35e75db None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:29.521496 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c96299-7798-4ca8-ba53-e9e1d35e75db None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:29.523109 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 43/86] 10.222.0.64 () {56 vars in 1108 bytes} [Tue Aug 2 09:45:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:45:29.525131 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c96299-7798-4ca8-ba53-e9e1d35e75db None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:29.542340 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:29.575008 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3207d50c-1c8b-46d0-91af-e7859cddfe5e None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:29.611453 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08b8648-97d7-42d2-89c8-1e3e8aaebef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:29.612372 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08b8648-97d7-42d2-89c8-1e3e8aaebef0 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:29.612820 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08b8648-97d7-42d2-89c8-1e3e8aaebef0 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:29.613301 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08b8648-97d7-42d2-89c8-1e3e8aaebef0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:29.635877 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c96299-7798-4ca8-ba53-e9e1d35e75db None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:29.674753 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d33bda8-29b9-41b9-9855-711a85e0617c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:29.675411 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d33bda8-29b9-41b9-9855-711a85e0617c None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:29.675727 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d33bda8-29b9-41b9-9855-711a85e0617c None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:29.676005 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d33bda8-29b9-41b9-9855-711a85e0617c None admin] PATH_INFO: `/v3/users/6b35b3486c4b4db6a107718cf2807616` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:29.677277 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d33bda8-29b9-41b9-9855-711a85e0617c None admin] RBAC: Authorizing `identity:get_user(user_id=6b35b3486c4b4db6a107718cf2807616)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:29.678751 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d33bda8-29b9-41b9-9855-711a85e0617c None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:29.680430 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 44/87] 10.222.0.64 () {56 vars in 1177 bytes} [Tue Aug 2 09:45:29 2022] GET /identity/v3/users/6b35b3486c4b4db6a107718cf2807616 => generated 275 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:45:29.681227 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d33bda8-29b9-41b9-9855-711a85e0617c None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:29.690643 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08b8648-97d7-42d2-89c8-1e3e8aaebef0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:29.693159 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08b8648-97d7-42d2-89c8-1e3e8aaebef0 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:29.724720 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:29.770853 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:29.789945 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d33bda8-29b9-41b9-9855-711a85e0617c None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:29.793656 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 44/88] 10.222.0.64 () {58 vars in 1282 bytes} [Tue Aug 2 09:45:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 09:45:29.794518 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08b8648-97d7-42d2-89c8-1e3e8aaebef0 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:29.823332 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05eb1200-f85f-4cba-8c55-a4eb329f4904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:29.823953 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eb1200-f85f-4cba-8c55-a4eb329f4904 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:29.824265 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eb1200-f85f-4cba-8c55-a4eb329f4904 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:29.824555 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eb1200-f85f-4cba-8c55-a4eb329f4904 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:29.825351 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05eb1200-f85f-4cba-8c55-a4eb329f4904 None admin] RBAC: Adding query filter params (role.id=66c1074780954f8e87ed11a5c4e8c36a, scope.project.id=ae78e3e21c0448c0b412e161cc374652, user.id=6b35b3486c4b4db6a107718cf2807616) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:45:29.825665 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05eb1200-f85f-4cba-8c55-a4eb329f4904 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:29.827131 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05eb1200-f85f-4cba-8c55-a4eb329f4904 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:29.836339 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 45/89] 10.222.0.64 () {56 vars in 1374 bytes} [Tue Aug 2 09:45:29 2022] GET /identity/v3/role_assignments?user.id=6b35b3486c4b4db6a107718cf2807616&scope.project.id=ae78e3e21c0448c0b412e161cc374652&role.id=66c1074780954f8e87ed11a5c4e8c36a => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:45:29.837147 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05eb1200-f85f-4cba-8c55-a4eb329f4904 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:29.918378 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08b8648-97d7-42d2-89c8-1e3e8aaebef0 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:29.956649 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:29.961489 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05eb1200-f85f-4cba-8c55-a4eb329f4904 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:29.962700 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fad5eb-d106-4bbc-a466-1e887c14dd96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:29.963682 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fad5eb-d106-4bbc-a466-1e887c14dd96 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:29.964199 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fad5eb-d106-4bbc-a466-1e887c14dd96 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:29.964512 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fad5eb-d106-4bbc-a466-1e887c14dd96 None admin] PATH_INFO: `/v3/projects/ae78e3e21c0448c0b412e161cc374652` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:29.965973 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fad5eb-d106-4bbc-a466-1e887c14dd96 None admin] RBAC: Authorizing `identity:get_project(project_id=ae78e3e21c0448c0b412e161cc374652)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:29.967516 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fad5eb-d106-4bbc-a466-1e887c14dd96 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:29.969512 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 45/90] 10.222.0.64 () {56 vars in 1186 bytes} [Tue Aug 2 09:45:29 2022] GET /identity/v3/projects/ae78e3e21c0448c0b412e161cc374652 => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:45:29.970267 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fad5eb-d106-4bbc-a466-1e887c14dd96 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:30.009075 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2ca724-490d-4938-8403-9056b501f8b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:30.009940 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2ca724-490d-4938-8403-9056b501f8b6 None admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:30.010230 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2ca724-490d-4938-8403-9056b501f8b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:30.010513 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2ca724-490d-4938-8403-9056b501f8b6 None admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:30.011218 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2ca724-490d-4938-8403-9056b501f8b6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:30.012734 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2ca724-490d-4938-8403-9056b501f8b6 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:30.044096 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 46/91] 10.222.0.64 () {60 vars in 1132 bytes} [Tue Aug 2 09:45:29 2022] POST /identity/v3/users => generated 289 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:45:30.045412 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2ca724-490d-4938-8403-9056b501f8b6 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:30.055736 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:30.115538 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fad5eb-d106-4bbc-a466-1e887c14dd96 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:30.163047 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbfa86f-97cc-4e87-9549-5e545fbf4399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:30.163823 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbfa86f-97cc-4e87-9549-5e545fbf4399 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:30.164232 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbfa86f-97cc-4e87-9549-5e545fbf4399 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:30.164614 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbfa86f-97cc-4e87-9549-5e545fbf4399 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:30.221805 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2ca724-490d-4938-8403-9056b501f8b6 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:30.271221 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:30.274903 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbfa86f-97cc-4e87-9549-5e545fbf4399 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:30.276649 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbfa86f-97cc-4e87-9549-5e545fbf4399 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:30.277305 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625b66b5-7e62-4987-9847-b92eb0228713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:30.277981 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625b66b5-7e62-4987-9847-b92eb0228713 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:30.278275 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625b66b5-7e62-4987-9847-b92eb0228713 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:30.278559 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625b66b5-7e62-4987-9847-b92eb0228713 None admin] PATH_INFO: `/v3/projects/ae78e3e21c0448c0b412e161cc374652` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:30.280119 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625b66b5-7e62-4987-9847-b92eb0228713 None admin] RBAC: Authorizing `identity:get_project(project_id=ae78e3e21c0448c0b412e161cc374652)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:30.282860 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625b66b5-7e62-4987-9847-b92eb0228713 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:30.289598 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 47/92] 10.222.0.64 () {56 vars in 1186 bytes} [Tue Aug 2 09:45:30 2022] GET /identity/v3/projects/ae78e3e21c0448c0b412e161cc374652 => generated 298 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:45:30.289598 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625b66b5-7e62-4987-9847-b92eb0228713 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:30.390207 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:30.430200 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 46/93] 10.222.0.64 () {58 vars in 1282 bytes} [Tue Aug 2 09:45:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 09:45:30.430200 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbfa86f-97cc-4e87-9549-5e545fbf4399 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:30.434327 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625b66b5-7e62-4987-9847-b92eb0228713 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:30.480786 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cae2720-e923-42c3-a2a1-8822fa8f01e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:30.481787 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cae2720-e923-42c3-a2a1-8822fa8f01e7 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:30.482265 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cae2720-e923-42c3-a2a1-8822fa8f01e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:30.482718 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cae2720-e923-42c3-a2a1-8822fa8f01e7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:30.483708 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cae2720-e923-42c3-a2a1-8822fa8f01e7 None admin] RBAC: Adding query filter params (role.id=06dfd237063c42a4bff1ad8ca29c07b0, scope.project.id=ae78e3e21c0448c0b412e161cc374652, user.id=6b35b3486c4b4db6a107718cf2807616) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:45:30.484276 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cae2720-e923-42c3-a2a1-8822fa8f01e7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:30.485906 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cae2720-e923-42c3-a2a1-8822fa8f01e7 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:30.494101 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 48/94] 10.222.0.64 () {56 vars in 1374 bytes} [Tue Aug 2 09:45:30 2022] GET /identity/v3/role_assignments?user.id=6b35b3486c4b4db6a107718cf2807616&scope.project.id=ae78e3e21c0448c0b412e161cc374652&role.id=06dfd237063c42a4bff1ad8ca29c07b0 => generated 259 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:45:30.495295 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cae2720-e923-42c3-a2a1-8822fa8f01e7 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:30.558241 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:30.561929 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbfa86f-97cc-4e87-9549-5e545fbf4399 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:30.605313 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8152cc9-f66c-41f1-80ca-fc078167a3bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:30.606264 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8152cc9-f66c-41f1-80ca-fc078167a3bf None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:30.606760 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8152cc9-f66c-41f1-80ca-fc078167a3bf None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:30.607212 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8152cc9-f66c-41f1-80ca-fc078167a3bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:30.608216 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8152cc9-f66c-41f1-80ca-fc078167a3bf None admin] RBAC: Adding query filter params (role.id=1000cf318fa14915b8091599eaead5de, scope.project.id=ae78e3e21c0448c0b412e161cc374652, user.id=5c5f9105c00b4d00a6249c76315d5e3e) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:45:30.608742 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8152cc9-f66c-41f1-80ca-fc078167a3bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:30.610444 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8152cc9-f66c-41f1-80ca-fc078167a3bf None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:30.619044 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 47/95] 10.222.0.64 () {56 vars in 1374 bytes} [Tue Aug 2 09:45:30 2022] GET /identity/v3/role_assignments?user.id=5c5f9105c00b4d00a6249c76315d5e3e&scope.project.id=ae78e3e21c0448c0b412e161cc374652&role.id=1000cf318fa14915b8091599eaead5de => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:45:30.647892 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cae2720-e923-42c3-a2a1-8822fa8f01e7 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:30.699461 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673254e7-e29d-496e-b0a3-ef9146254f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:30.700530 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673254e7-e29d-496e-b0a3-ef9146254f69 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:30.700986 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673254e7-e29d-496e-b0a3-ef9146254f69 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:30.701458 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673254e7-e29d-496e-b0a3-ef9146254f69 None admin] PATH_INFO: `/v3/projects/94a149b343344045872ab5da9199fc56` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:30.708481 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673254e7-e29d-496e-b0a3-ef9146254f69 None admin] RBAC: Authorizing `identity:get_project(project_id=94a149b343344045872ab5da9199fc56)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:30.710320 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673254e7-e29d-496e-b0a3-ef9146254f69 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:30.712452 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 49/96] 10.222.0.64 () {56 vars in 1186 bytes} [Tue Aug 2 09:45:30 2022] GET /identity/v3/projects/94a149b343344045872ab5da9199fc56 => generated 312 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:45:30.719046 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8152cc9-f66c-41f1-80ca-fc078167a3bf None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:30.860568 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8152cc9-f66c-41f1-80ca-fc078167a3bf None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:30.914210 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6dcef7-6829-425f-89dc-d7c14adc6dbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:30.914858 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6dcef7-6829-425f-89dc-d7c14adc6dbc None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:30.915192 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6dcef7-6829-425f-89dc-d7c14adc6dbc None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:30.915477 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6dcef7-6829-425f-89dc-d7c14adc6dbc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:30.916340 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6dcef7-6829-425f-89dc-d7c14adc6dbc None admin] RBAC: Adding query filter params (role.id=66c1074780954f8e87ed11a5c4e8c36a, scope.project.id=94a149b343344045872ab5da9199fc56, user.id=6b35b3486c4b4db6a107718cf2807616) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:45:30.916662 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6dcef7-6829-425f-89dc-d7c14adc6dbc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:30.918162 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6dcef7-6829-425f-89dc-d7c14adc6dbc None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:30.929875 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 48/97] 10.222.0.64 () {56 vars in 1374 bytes} [Tue Aug 2 09:45:30 2022] GET /identity/v3/role_assignments?user.id=6b35b3486c4b4db6a107718cf2807616&scope.project.id=94a149b343344045872ab5da9199fc56&role.id=66c1074780954f8e87ed11a5c4e8c36a => generated 259 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:45:32.403781 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b8f38d-94e4-4aa9-aba6-7e1b0726af5c None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:32.404311 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b8f38d-94e4-4aa9-aba6-7e1b0726af5c None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:32.404387 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b8f38d-94e4-4aa9-aba6-7e1b0726af5c None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:32.405743 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 50/98] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:45:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:45:32.411392 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316436cd-bff2-45c8-88d9-c349cef10010 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:32.411840 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316436cd-bff2-45c8-88d9-c349cef10010 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:32.412019 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316436cd-bff2-45c8-88d9-c349cef10010 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:32.440939 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-316436cd-bff2-45c8-88d9-c349cef10010 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:45:32.442746 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-316436cd-bff2-45c8-88d9-c349cef10010 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:45:32.471272 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:32.524179 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 49/99] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:45:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 09:45:32.546244 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b8f38d-94e4-4aa9-aba6-7e1b0726af5c None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:32.552804 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:32.689325 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b8f38d-94e4-4aa9-aba6-7e1b0726af5c None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:32.739320 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2df8da-bdd0-479f-9e94-ad1033c7bffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:32.740372 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2df8da-bdd0-479f-9e94-ad1033c7bffd None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:32.740857 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2df8da-bdd0-479f-9e94-ad1033c7bffd None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:32.741329 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2df8da-bdd0-479f-9e94-ad1033c7bffd None admin] PATH_INFO: `/v3/roles/member` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:32.761218 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2df8da-bdd0-479f-9e94-ad1033c7bffd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 09:45:32.767083 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-8a2df8da-bdd0-479f-9e94-ad1033c7bffd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 02 09:45:32.767083 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:45:32.767083 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:45:32.767083 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:45:32.767083 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:45:32.767083 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:45:32.767083 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:45:32.767083 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:45:32.767083 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:45:32.767083 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:45:32.767083 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:45:32.767083 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:45:32.767083 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 09:45:32.767083 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 09:45:32.767083 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 09:45:32.767083 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 09:45:32.767950 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 09:45:32.767950 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 09:45:32.767950 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 09:45:32.767950 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 09:45:32.767950 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:45:32.767950 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:45:32.767950 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:45:32.767950 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:45:32.767950 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:45:32.767950 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:45:32.767950 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:45:32.767950 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:45:32.767950 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:45:32.767950 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:45:32.767950 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:45:32.767950 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:45:32.768762 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:45:32.768762 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:45:32.768762 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:45:32.768762 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:45:32.768762 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 09:45:32.768762 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 09:45:32.768762 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 09:45:32.768762 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 09:45:32.768762 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 09:45:32.768762 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 09:45:32.768762 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 02 09:45:32.768762 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:45:32.768762 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 51/100] 10.222.0.64 () {56 vars in 1099 bytes} [Tue Aug 2 09:45:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:45:32.776198 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316436cd-bff2-45c8-88d9-c349cef10010 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:32.910973 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316436cd-bff2-45c8-88d9-c349cef10010 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:32.954194 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6e5ded-553a-4563-bbe3-5169ac2306e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:32.955045 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6e5ded-553a-4563-bbe3-5169ac2306e4 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:32.955485 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6e5ded-553a-4563-bbe3-5169ac2306e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:32.955914 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6e5ded-553a-4563-bbe3-5169ac2306e4 None admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:32.956857 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6e5ded-553a-4563-bbe3-5169ac2306e4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:45:32.960639 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6e5ded-553a-4563-bbe3-5169ac2306e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:32.962210 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6e5ded-553a-4563-bbe3-5169ac2306e4 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:32.971122 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 50/101] 10.222.0.64 () {56 vars in 1101 bytes} [Tue Aug 2 09:45:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:45:32.977675 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2df8da-bdd0-479f-9e94-ad1033c7bffd None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:33.041514 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd5889b-6416-483f-9de2-13b810579b51 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:33.042059 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd5889b-6416-483f-9de2-13b810579b51 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:33.042493 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd5889b-6416-483f-9de2-13b810579b51 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:33.043864 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 51/102] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:45:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:45:33.061134 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bc8309-1a34-41c8-a29c-1cbf66f72a7b None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:33.061749 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bc8309-1a34-41c8-a29c-1cbf66f72a7b None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:33.062132 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bc8309-1a34-41c8-a29c-1cbf66f72a7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:33.088612 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:33.089574 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-a3bc8309-1a34-41c8-a29c-1cbf66f72a7b None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:45:33.090775 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-a3bc8309-1a34-41c8-a29c-1cbf66f72a7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:45:33.116379 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:33.177751 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2df8da-bdd0-479f-9e94-ad1033c7bffd None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:33.207229 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 52/103] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:45:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 09:45:33.245938 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7fc960-bc0b-478c-a04a-2145a260003f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:33.246605 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7fc960-bc0b-478c-a04a-2145a260003f None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:33.246902 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7fc960-bc0b-478c-a04a-2145a260003f None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:33.247183 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7fc960-bc0b-478c-a04a-2145a260003f None admin] PATH_INFO: `/v3/users/6b35b3486c4b4db6a107718cf2807616` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:33.253107 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7fc960-bc0b-478c-a04a-2145a260003f None admin] RBAC: Authorizing `identity:get_user(user_id=6b35b3486c4b4db6a107718cf2807616)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:33.254591 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7fc960-bc0b-478c-a04a-2145a260003f None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:33.256374 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 52/104] 10.222.0.64 () {56 vars in 1177 bytes} [Tue Aug 2 09:45:32 2022] GET /identity/v3/users/6b35b3486c4b4db6a107718cf2807616 => generated 275 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:45:33.261791 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bc8309-1a34-41c8-a29c-1cbf66f72a7b None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:33.263998 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7fc960-bc0b-478c-a04a-2145a260003f None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:33.404991 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7fc960-bc0b-478c-a04a-2145a260003f None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:33.440731 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bc8309-1a34-41c8-a29c-1cbf66f72a7b None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:33.473240 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:33.489599 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ded591a-f45b-47ea-984d-efeb1e84f6ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:33.490276 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ded591a-f45b-47ea-984d-efeb1e84f6ed None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:33.490574 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ded591a-f45b-47ea-984d-efeb1e84f6ed None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:33.490862 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ded591a-f45b-47ea-984d-efeb1e84f6ed None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:33.521197 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ded591a-f45b-47ea-984d-efeb1e84f6ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 09:45:33.526996 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-1ded591a-f45b-47ea-984d-efeb1e84f6ed None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 02 09:45:33.526996 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:45:33.526996 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:45:33.526996 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:45:33.526996 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:45:33.526996 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:45:33.526996 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:45:33.526996 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:45:33.526996 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:45:33.526996 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:45:33.526996 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:45:33.526996 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:45:33.526996 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 09:45:33.526996 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 09:45:33.526996 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 09:45:33.526996 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 09:45:33.527842 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 09:45:33.527842 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 09:45:33.527842 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 09:45:33.527842 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 09:45:33.527842 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:45:33.527842 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:45:33.527842 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:45:33.527842 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:45:33.527842 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:45:33.527842 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:45:33.527842 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:45:33.527842 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:45:33.527842 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:45:33.527842 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:45:33.527842 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:45:33.527842 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:45:33.528655 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:45:33.528655 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:45:33.528655 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:45:33.528655 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:45:33.528655 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 09:45:33.528655 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 09:45:33.528655 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 09:45:33.528655 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 09:45:33.528655 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 09:45:33.528655 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 09:45:33.528655 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 02 09:45:33.528655 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:45:33.528655 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 53/105] 10.222.0.64 () {56 vars in 1099 bytes} [Tue Aug 2 09:45:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 09:45:33.530091 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7af9c2-b427-4a9f-9a6f-1574d11b0da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:33.530762 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7af9c2-b427-4a9f-9a6f-1574d11b0da3 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:33.531136 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7af9c2-b427-4a9f-9a6f-1574d11b0da3 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:33.531397 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7af9c2-b427-4a9f-9a6f-1574d11b0da3 None admin] PATH_INFO: `/v3/projects/ae78e3e21c0448c0b412e161cc374652` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:33.535787 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7af9c2-b427-4a9f-9a6f-1574d11b0da3 None admin] RBAC: Authorizing `identity:get_project(project_id=ae78e3e21c0448c0b412e161cc374652)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:33.536367 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ded591a-f45b-47ea-984d-efeb1e84f6ed None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:33.537341 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7af9c2-b427-4a9f-9a6f-1574d11b0da3 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:33.539383 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 53/106] 10.222.0.64 () {56 vars in 1186 bytes} [Tue Aug 2 09:45:33 2022] GET /identity/v3/projects/ae78e3e21c0448c0b412e161cc374652 => generated 298 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:45:33.544647 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7af9c2-b427-4a9f-9a6f-1574d11b0da3 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:33.592601 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:33.747678 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7af9c2-b427-4a9f-9a6f-1574d11b0da3 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:33.761000 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ded591a-f45b-47ea-984d-efeb1e84f6ed None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:33.786609 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6673368-59d7-46b6-8b28-318a86e0b8f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:33.787290 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6673368-59d7-46b6-8b28-318a86e0b8f1 None admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:33.787677 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6673368-59d7-46b6-8b28-318a86e0b8f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:33.787954 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6673368-59d7-46b6-8b28-318a86e0b8f1 None admin] PATH_INFO: `/v3/projects/ae78e3e21c0448c0b412e161cc374652/users/6b35b3486c4b4db6a107718cf2807616/roles/66c1074780954f8e87ed11a5c4e8c36a` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:33.790083 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6673368-59d7-46b6-8b28-318a86e0b8f1 None admin] RBAC: Authorizing `identity:create_grant(project_id=ae78e3e21c0448c0b412e161cc374652, user_id=6b35b3486c4b4db6a107718cf2807616, role_id=66c1074780954f8e87ed11a5c4e8c36a)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:33.791809 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6673368-59d7-46b6-8b28-318a86e0b8f1 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:33.799792 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ca74c5-68cb-47f8-91f1-2da8dc7afc4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:33.800510 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ca74c5-68cb-47f8-91f1-2da8dc7afc4d None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:33.801185 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ca74c5-68cb-47f8-91f1-2da8dc7afc4d None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:33.801473 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ca74c5-68cb-47f8-91f1-2da8dc7afc4d None admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:33.802357 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ca74c5-68cb-47f8-91f1-2da8dc7afc4d None admin] RBAC: Adding query filter params (name=reader) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:45:33.802724 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ca74c5-68cb-47f8-91f1-2da8dc7afc4d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:33.804390 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ca74c5-68cb-47f8-91f1-2da8dc7afc4d None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:33.806975 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 54/107] 10.222.0.64 () {58 vars in 1439 bytes} [Tue Aug 2 09:45:33 2022] PUT /identity/v3/projects/ae78e3e21c0448c0b412e161cc374652/users/6b35b3486c4b4db6a107718cf2807616/roles/66c1074780954f8e87ed11a5c4e8c36a => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:45:33.808383 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baef2c39-3355-4f21-9784-2a09ef7b5eb2 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:33.808708 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baef2c39-3355-4f21-9784-2a09ef7b5eb2 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:33.808999 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baef2c39-3355-4f21-9784-2a09ef7b5eb2 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:33.810502 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 55/108] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:45:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:45:33.811843 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fda02c9-5712-4a81-98b0-00557e021b1f None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:33.812258 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fda02c9-5712-4a81-98b0-00557e021b1f None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:33.812539 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fda02c9-5712-4a81-98b0-00557e021b1f None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:33.813833 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 56/109] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:45:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:45:33.817368 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 54/110] 10.222.0.64 () {56 vars in 1101 bytes} [Tue Aug 2 09:45:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:45:33.819527 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e0130e-2706-42cc-bf78-032ed5641db8 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:33.819895 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e0130e-2706-42cc-bf78-032ed5641db8 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:33.820269 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e0130e-2706-42cc-bf78-032ed5641db8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:33.823859 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e617cac3-948d-43dd-b03a-0062129d798e None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:33.824184 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e617cac3-948d-43dd-b03a-0062129d798e None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:33.824465 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e617cac3-948d-43dd-b03a-0062129d798e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:33.844793 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-e617cac3-948d-43dd-b03a-0062129d798e None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:45:33.845037 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-25e0130e-2706-42cc-bf78-032ed5641db8 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:45:33.846021 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-e617cac3-948d-43dd-b03a-0062129d798e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:45:33.846195 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-25e0130e-2706-42cc-bf78-032ed5641db8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:45:33.913573 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:33.939075 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 55/111] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:45:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 09:45:33.940282 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e617cac3-948d-43dd-b03a-0062129d798e None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:33.946043 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 57/112] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:45:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 09:45:33.947652 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6a4d10-6dd6-4ded-8394-68f4f334dce8 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:33.948091 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6a4d10-6dd6-4ded-8394-68f4f334dce8 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:33.948674 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6a4d10-6dd6-4ded-8394-68f4f334dce8 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:33.951331 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 58/113] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:45:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:45:33.963209 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b7bee8-c80d-468a-accf-4e653681966a None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:33.963957 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b7bee8-c80d-468a-accf-4e653681966a None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:33.964595 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b7bee8-c80d-468a-accf-4e653681966a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:33.996151 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-05b7bee8-c80d-468a-accf-4e653681966a None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:45:33.998015 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-05b7bee8-c80d-468a-accf-4e653681966a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:45:34.003587 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:34.079593 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e617cac3-948d-43dd-b03a-0062129d798e None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:34.086278 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 59/114] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:45:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 09:45:34.089491 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b7bee8-c80d-468a-accf-4e653681966a None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:34.140707 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cba9c8-7c63-448c-9048-c4ec34cd9497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:34.141603 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cba9c8-7c63-448c-9048-c4ec34cd9497 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:34.142148 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cba9c8-7c63-448c-9048-c4ec34cd9497 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:34.142574 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cba9c8-7c63-448c-9048-c4ec34cd9497 None admin] PATH_INFO: `/v3/users/c7f7673261604df6a51226b783e563b5` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:34.161616 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cba9c8-7c63-448c-9048-c4ec34cd9497 None admin] RBAC: Authorizing `identity:get_user(user_id=c7f7673261604df6a51226b783e563b5)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:34.163943 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cba9c8-7c63-448c-9048-c4ec34cd9497 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:34.166408 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 56/115] 10.222.0.64 () {56 vars in 1177 bytes} [Tue Aug 2 09:45:33 2022] GET /identity/v3/users/c7f7673261604df6a51226b783e563b5 => generated 289 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:45:34.167531 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cba9c8-7c63-448c-9048-c4ec34cd9497 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:34.202956 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b7bee8-c80d-468a-accf-4e653681966a None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:34.215315 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:34.240692 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7656c1-3426-4919-acfe-e8afaba4e40f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:34.241587 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7656c1-3426-4919-acfe-e8afaba4e40f None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:34.242019 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7656c1-3426-4919-acfe-e8afaba4e40f None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:34.242438 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7656c1-3426-4919-acfe-e8afaba4e40f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:34.250273 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:34.254532 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7656c1-3426-4919-acfe-e8afaba4e40f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 09:45:34.260043 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-3c7656c1-3426-4919-acfe-e8afaba4e40f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 09:45:34.260043 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:45:34.260043 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:45:34.260043 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:45:34.260043 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:45:34.260043 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:45:34.260043 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:45:34.260043 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:45:34.260043 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:45:34.260043 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:45:34.260043 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:45:34.260043 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:45:34.260043 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 09:45:34.260043 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 09:45:34.260043 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 09:45:34.260043 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 09:45:34.261095 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 09:45:34.261095 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 09:45:34.261095 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 09:45:34.261095 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 09:45:34.261095 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:45:34.261095 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:45:34.261095 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:45:34.261095 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:45:34.261095 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:45:34.261095 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:45:34.261095 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:45:34.261095 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:45:34.261095 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:45:34.261095 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:45:34.261095 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:45:34.261095 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:45:34.262048 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:45:34.262048 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:45:34.262048 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:45:34.262048 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:45:34.262048 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 09:45:34.262048 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 09:45:34.262048 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 09:45:34.262048 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 09:45:34.262048 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 09:45:34.262048 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 09:45:34.262048 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 09:45:34.262048 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:45:34.263934 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 60/116] 10.222.0.64 () {56 vars in 1096 bytes} [Tue Aug 2 09:45:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:45:34.264908 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7656c1-3426-4919-acfe-e8afaba4e40f None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:34.283179 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cba9c8-7c63-448c-9048-c4ec34cd9497 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:34.323974 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450b7e9b-9f30-4db1-85f2-193b13d1b2db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:34.324852 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450b7e9b-9f30-4db1-85f2-193b13d1b2db None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:34.325150 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450b7e9b-9f30-4db1-85f2-193b13d1b2db None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:34.325435 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450b7e9b-9f30-4db1-85f2-193b13d1b2db None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:34.338585 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450b7e9b-9f30-4db1-85f2-193b13d1b2db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 09:45:34.343979 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-450b7e9b-9f30-4db1-85f2-193b13d1b2db None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 09:45:34.343979 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:45:34.343979 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:45:34.343979 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:45:34.343979 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:45:34.343979 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:45:34.343979 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:45:34.343979 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:45:34.343979 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:45:34.343979 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:45:34.343979 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:45:34.343979 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:45:34.343979 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 09:45:34.343979 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 09:45:34.343979 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 09:45:34.343979 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 09:45:34.345074 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 09:45:34.345074 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 09:45:34.345074 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 09:45:34.345074 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 09:45:34.345074 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:45:34.345074 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:45:34.345074 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:45:34.345074 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:45:34.345074 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:45:34.345074 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:45:34.345074 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:45:34.345074 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:45:34.345074 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:45:34.345074 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:45:34.345074 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:45:34.345074 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:45:34.345899 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:45:34.345899 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:45:34.345899 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:45:34.345899 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:45:34.345899 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 09:45:34.345899 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 09:45:34.345899 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 09:45:34.345899 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 09:45:34.345899 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 09:45:34.345899 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 09:45:34.345899 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 09:45:34.345899 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:45:34.345899 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 57/117] 10.222.0.64 () {56 vars in 1114 bytes} [Tue Aug 2 09:45:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:45:34.346658 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450b7e9b-9f30-4db1-85f2-193b13d1b2db None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:34.375948 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7656c1-3426-4919-acfe-e8afaba4e40f None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:34.414365 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679a9a94-b77d-46e0-996b-0fb321752627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:34.415303 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679a9a94-b77d-46e0-996b-0fb321752627 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:34.415847 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679a9a94-b77d-46e0-996b-0fb321752627 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:34.416320 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679a9a94-b77d-46e0-996b-0fb321752627 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:34.417760 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679a9a94-b77d-46e0-996b-0fb321752627 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:34.419461 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679a9a94-b77d-46e0-996b-0fb321752627 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:34.421540 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 61/118] 10.222.0.64 () {56 vars in 1108 bytes} [Tue Aug 2 09:45:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:45:34.422490 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679a9a94-b77d-46e0-996b-0fb321752627 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:34.459559 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:34.462564 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450b7e9b-9f30-4db1-85f2-193b13d1b2db None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:34.499703 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:34.501041 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b86e19-061a-4e4d-ad43-b80796282530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:34.501624 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b86e19-061a-4e4d-ad43-b80796282530 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:34.501943 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b86e19-061a-4e4d-ad43-b80796282530 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:34.502224 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b86e19-061a-4e4d-ad43-b80796282530 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:34.539408 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679a9a94-b77d-46e0-996b-0fb321752627 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:34.574692 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b86e19-061a-4e4d-ad43-b80796282530 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:34.575328 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc4436a-ece3-4c10-adef-9062ac5097a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:34.575958 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc4436a-ece3-4c10-adef-9062ac5097a5 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:34.576202 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b86e19-061a-4e4d-ad43-b80796282530 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:34.576330 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc4436a-ece3-4c10-adef-9062ac5097a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:34.576564 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc4436a-ece3-4c10-adef-9062ac5097a5 None admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:34.577310 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc4436a-ece3-4c10-adef-9062ac5097a5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:45:34.577627 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc4436a-ece3-4c10-adef-9062ac5097a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:34.579021 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc4436a-ece3-4c10-adef-9062ac5097a5 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:34.587825 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 62/119] 10.222.0.64 () {56 vars in 1099 bytes} [Tue Aug 2 09:45:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:45:34.588700 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc4436a-ece3-4c10-adef-9062ac5097a5 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:34.680971 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:34.689608 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 58/120] 10.222.0.64 () {58 vars in 1282 bytes} [Tue Aug 2 09:45:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 09:45:34.689608 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b86e19-061a-4e4d-ad43-b80796282530 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:34.700584 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc4436a-ece3-4c10-adef-9062ac5097a5 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:34.729603 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:34.739313 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c609fc60-f5a8-42af-8538-2ef6d22cd610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:34.739941 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c609fc60-f5a8-42af-8538-2ef6d22cd610 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:34.740231 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c609fc60-f5a8-42af-8538-2ef6d22cd610 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:34.740546 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c609fc60-f5a8-42af-8538-2ef6d22cd610 None admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:34.741626 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c609fc60-f5a8-42af-8538-2ef6d22cd610 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:45:34.741626 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c609fc60-f5a8-42af-8538-2ef6d22cd610 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:34.743093 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c609fc60-f5a8-42af-8538-2ef6d22cd610 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:34.751472 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 63/121] 10.222.0.64 () {56 vars in 1111 bytes} [Tue Aug 2 09:45:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:45:34.756445 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c94e3f-0842-4f6a-b2ab-22213402ff9e None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:34.756445 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c94e3f-0842-4f6a-b2ab-22213402ff9e None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:34.756445 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c94e3f-0842-4f6a-b2ab-22213402ff9e None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:34.756445 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 64/122] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:45:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:45:34.756445 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c94e3f-0842-4f6a-b2ab-22213402ff9e None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:34.803470 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b86e19-061a-4e4d-ad43-b80796282530 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:34.844998 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ece9d4-b4d0-43db-9b86-241d23c70b67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:34.845637 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ece9d4-b4d0-43db-9b86-241d23c70b67 None admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:34.845987 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ece9d4-b4d0-43db-9b86-241d23c70b67 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:34.846375 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ece9d4-b4d0-43db-9b86-241d23c70b67 None admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:34.847623 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ece9d4-b4d0-43db-9b86-241d23c70b67 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:34.849148 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ece9d4-b4d0-43db-9b86-241d23c70b67 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:34.867190 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c94e3f-0842-4f6a-b2ab-22213402ff9e None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:34.888278 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 59/123] 10.222.0.64 () {60 vars in 1141 bytes} [Tue Aug 2 09:45:34 2022] POST /identity/v3/projects => generated 302 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:45:34.888278 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ece9d4-b4d0-43db-9b86-241d23c70b67 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:34.924023 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032d3177-d729-4a55-ae5e-7fdee6d54f03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:34.925523 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032d3177-d729-4a55-ae5e-7fdee6d54f03 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:34.926080 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032d3177-d729-4a55-ae5e-7fdee6d54f03 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:34.926700 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032d3177-d729-4a55-ae5e-7fdee6d54f03 None admin] PATH_INFO: `/v3/users/5c5f9105c00b4d00a6249c76315d5e3e` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:34.928453 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032d3177-d729-4a55-ae5e-7fdee6d54f03 None admin] RBAC: Authorizing `identity:get_user(user_id=5c5f9105c00b4d00a6249c76315d5e3e)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:34.930293 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032d3177-d729-4a55-ae5e-7fdee6d54f03 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:34.932469 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 65/124] 10.222.0.64 () {56 vars in 1177 bytes} [Tue Aug 2 09:45:34 2022] GET /identity/v3/users/5c5f9105c00b4d00a6249c76315d5e3e => generated 247 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:45:34.933472 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032d3177-d729-4a55-ae5e-7fdee6d54f03 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:34.968061 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:35.032112 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:35.050395 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ece9d4-b4d0-43db-9b86-241d23c70b67 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:35.097553 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8305416-a164-47c4-8aaf-34fcb5b8400d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:35.098218 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8305416-a164-47c4-8aaf-34fcb5b8400d None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:35.098555 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8305416-a164-47c4-8aaf-34fcb5b8400d None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:35.098737 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032d3177-d729-4a55-ae5e-7fdee6d54f03 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:35.098845 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8305416-a164-47c4-8aaf-34fcb5b8400d None admin] PATH_INFO: `/v3/projects/ae78e3e21c0448c0b412e161cc374652` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:35.100203 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8305416-a164-47c4-8aaf-34fcb5b8400d None admin] RBAC: Authorizing `identity:get_project(project_id=ae78e3e21c0448c0b412e161cc374652)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:35.101889 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8305416-a164-47c4-8aaf-34fcb5b8400d None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:35.103882 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 60/125] 10.222.0.64 () {56 vars in 1186 bytes} [Tue Aug 2 09:45:34 2022] GET /identity/v3/projects/ae78e3e21c0448c0b412e161cc374652 => generated 298 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:45:35.105046 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8727492-52ff-430a-9e6b-9a049770fc67 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:35.107623 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8727492-52ff-430a-9e6b-9a049770fc67 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:35.107623 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8727492-52ff-430a-9e6b-9a049770fc67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:35.125781 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-a8727492-52ff-430a-9e6b-9a049770fc67 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:45:35.126994 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-a8727492-52ff-430a-9e6b-9a049770fc67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:45:35.155672 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b615b29b-85f3-4b5e-b6bc-b2ec739ea2b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:35.156135 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b615b29b-85f3-4b5e-b6bc-b2ec739ea2b6 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:35.156370 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b615b29b-85f3-4b5e-b6bc-b2ec739ea2b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:35.156739 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b615b29b-85f3-4b5e-b6bc-b2ec739ea2b6 None admin] PATH_INFO: `/v3/users/6b35b3486c4b4db6a107718cf2807616` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:35.158331 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b615b29b-85f3-4b5e-b6bc-b2ec739ea2b6 None admin] RBAC: Authorizing `identity:get_user(user_id=6b35b3486c4b4db6a107718cf2807616)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:35.159788 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b615b29b-85f3-4b5e-b6bc-b2ec739ea2b6 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:35.161859 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 66/126] 10.222.0.64 () {56 vars in 1177 bytes} [Tue Aug 2 09:45:34 2022] GET /identity/v3/users/6b35b3486c4b4db6a107718cf2807616 => generated 275 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:45:35.162728 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b615b29b-85f3-4b5e-b6bc-b2ec739ea2b6 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:35.212800 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 61/127] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:45:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 09:45:35.213991 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8727492-52ff-430a-9e6b-9a049770fc67 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:35.279727 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:35.298251 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b615b29b-85f3-4b5e-b6bc-b2ec739ea2b6 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:35.328320 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:35.346260 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6d5ec5-b91a-44bc-9086-0261a2fbd409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:35.346910 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6d5ec5-b91a-44bc-9086-0261a2fbd409 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:35.347226 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6d5ec5-b91a-44bc-9086-0261a2fbd409 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:35.347508 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6d5ec5-b91a-44bc-9086-0261a2fbd409 None admin] PATH_INFO: `/v3/projects/ae78e3e21c0448c0b412e161cc374652` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:35.348866 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6d5ec5-b91a-44bc-9086-0261a2fbd409 None admin] RBAC: Authorizing `identity:get_project(project_id=ae78e3e21c0448c0b412e161cc374652)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:35.350337 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6d5ec5-b91a-44bc-9086-0261a2fbd409 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:35.352391 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 67/128] 10.222.0.64 () {56 vars in 1186 bytes} [Tue Aug 2 09:45:35 2022] GET /identity/v3/projects/ae78e3e21c0448c0b412e161cc374652 => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:45:35.354382 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8727492-52ff-430a-9e6b-9a049770fc67 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:35.357788 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6d5ec5-b91a-44bc-9086-0261a2fbd409 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:35.405835 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20781a5-9ed5-4950-9153-df4c96f4ac7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:35.407056 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20781a5-9ed5-4950-9153-df4c96f4ac7c None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:35.408017 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20781a5-9ed5-4950-9153-df4c96f4ac7c None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:35.408513 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20781a5-9ed5-4950-9153-df4c96f4ac7c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:35.410070 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20781a5-9ed5-4950-9153-df4c96f4ac7c None admin] RBAC: Adding query filter params (role.id=7fce0045dfa049bdb7ae0d2424de9f32, scope.project.id=ae78e3e21c0448c0b412e161cc374652, user.id=c7f7673261604df6a51226b783e563b5) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:45:35.411167 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20781a5-9ed5-4950-9153-df4c96f4ac7c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:35.415411 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20781a5-9ed5-4950-9153-df4c96f4ac7c None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:35.429267 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 62/129] 10.222.0.64 () {56 vars in 1374 bytes} [Tue Aug 2 09:45:35 2022] GET /identity/v3/role_assignments?user.id=c7f7673261604df6a51226b783e563b5&scope.project.id=ae78e3e21c0448c0b412e161cc374652&role.id=7fce0045dfa049bdb7ae0d2424de9f32 => generated 259 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:45:35.430584 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20781a5-9ed5-4950-9153-df4c96f4ac7c None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:35.527364 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6d5ec5-b91a-44bc-9086-0261a2fbd409 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:35.568496 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b8bb89-1247-4e50-8b37-865d9a74117d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:35.569735 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b8bb89-1247-4e50-8b37-865d9a74117d None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:35.570255 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b8bb89-1247-4e50-8b37-865d9a74117d None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:35.570740 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b8bb89-1247-4e50-8b37-865d9a74117d None admin] PATH_INFO: `/v3/projects/ae78e3e21c0448c0b412e161cc374652` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:35.572181 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b8bb89-1247-4e50-8b37-865d9a74117d None admin] RBAC: Authorizing `identity:get_project(project_id=ae78e3e21c0448c0b412e161cc374652)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:35.574917 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b8bb89-1247-4e50-8b37-865d9a74117d None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:35.576924 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 68/130] 10.222.0.64 () {56 vars in 1186 bytes} [Tue Aug 2 09:45:35 2022] GET /identity/v3/projects/ae78e3e21c0448c0b412e161cc374652 => generated 298 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:45:35.578026 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b8bb89-1247-4e50-8b37-865d9a74117d None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:35.653560 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:35.671222 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20781a5-9ed5-4950-9153-df4c96f4ac7c None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:35.692664 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:35.736502 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9d7624-7892-41a2-9e75-e16f8241b719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:35.737544 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9d7624-7892-41a2-9e75-e16f8241b719 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:35.738067 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9d7624-7892-41a2-9e75-e16f8241b719 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:35.738549 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9d7624-7892-41a2-9e75-e16f8241b719 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:35.752340 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9d7624-7892-41a2-9e75-e16f8241b719 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 09:45:35.754427 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b8bb89-1247-4e50-8b37-865d9a74117d None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:35.760953 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-ee9d7624-7892-41a2-9e75-e16f8241b719 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 02 09:45:35.760953 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:45:35.760953 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:45:35.760953 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:45:35.760953 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:45:35.760953 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:45:35.760953 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:45:35.760953 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:45:35.760953 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:45:35.760953 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:45:35.760953 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:45:35.760953 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:45:35.760953 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 09:45:35.760953 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 09:45:35.760953 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 09:45:35.760953 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 09:45:35.761919 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 09:45:35.761919 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 09:45:35.761919 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 09:45:35.761919 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 09:45:35.761919 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:45:35.761919 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:45:35.761919 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:45:35.761919 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:45:35.761919 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:45:35.761919 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:45:35.761919 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:45:35.761919 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:45:35.761919 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:45:35.761919 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:45:35.761919 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:45:35.761919 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:45:35.762736 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:45:35.762736 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:45:35.762736 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:45:35.762736 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:45:35.762736 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 09:45:35.762736 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 09:45:35.762736 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 09:45:35.762736 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 09:45:35.762736 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 09:45:35.762736 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 09:45:35.762736 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 02 09:45:35.762736 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:45:35.765249 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 63/131] 10.222.0.64 () {56 vars in 1099 bytes} [Tue Aug 2 09:45:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:45:35.767151 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9d7624-7892-41a2-9e75-e16f8241b719 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:35.819332 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f492a2-e4ad-4dba-bff5-20cfb192dd79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:35.820666 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f492a2-e4ad-4dba-bff5-20cfb192dd79 None admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:35.821263 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f492a2-e4ad-4dba-bff5-20cfb192dd79 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:35.821766 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f492a2-e4ad-4dba-bff5-20cfb192dd79 None admin] PATH_INFO: `/v3/projects/ae78e3e21c0448c0b412e161cc374652/users/5c5f9105c00b4d00a6249c76315d5e3e/roles/1000cf318fa14915b8091599eaead5de` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:35.833058 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f492a2-e4ad-4dba-bff5-20cfb192dd79 None admin] RBAC: Authorizing `identity:create_grant(project_id=ae78e3e21c0448c0b412e161cc374652, user_id=5c5f9105c00b4d00a6249c76315d5e3e, role_id=1000cf318fa14915b8091599eaead5de)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:35.836027 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f492a2-e4ad-4dba-bff5-20cfb192dd79 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:35.880516 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 69/132] 10.222.0.64 () {58 vars in 1439 bytes} [Tue Aug 2 09:45:35 2022] PUT /identity/v3/projects/ae78e3e21c0448c0b412e161cc374652/users/5c5f9105c00b4d00a6249c76315d5e3e/roles/1000cf318fa14915b8091599eaead5de => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:45:35.882035 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f492a2-e4ad-4dba-bff5-20cfb192dd79 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:35.905112 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9d7624-7892-41a2-9e75-e16f8241b719 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:35.973139 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5064c839-59fa-400f-939c-7f856b23d62f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:35.973888 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5064c839-59fa-400f-939c-7f856b23d62f None admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:35.974709 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5064c839-59fa-400f-939c-7f856b23d62f None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:35.975007 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5064c839-59fa-400f-939c-7f856b23d62f None admin] PATH_INFO: `/v3/projects/ae78e3e21c0448c0b412e161cc374652/users/6b35b3486c4b4db6a107718cf2807616/roles/06dfd237063c42a4bff1ad8ca29c07b0` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:35.981614 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5064c839-59fa-400f-939c-7f856b23d62f None admin] RBAC: Authorizing `identity:create_grant(project_id=ae78e3e21c0448c0b412e161cc374652, user_id=6b35b3486c4b4db6a107718cf2807616, role_id=06dfd237063c42a4bff1ad8ca29c07b0)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:35.983193 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5064c839-59fa-400f-939c-7f856b23d62f None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:36.007476 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 64/133] 10.222.0.64 () {58 vars in 1439 bytes} [Tue Aug 2 09:45:35 2022] PUT /identity/v3/projects/ae78e3e21c0448c0b412e161cc374652/users/6b35b3486c4b4db6a107718cf2807616/roles/06dfd237063c42a4bff1ad8ca29c07b0 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:45:36.172525 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f492a2-e4ad-4dba-bff5-20cfb192dd79 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:36.178271 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb6b85e-7d97-4258-9b37-d63790aedde5 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:36.178606 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb6b85e-7d97-4258-9b37-d63790aedde5 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:36.178846 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb6b85e-7d97-4258-9b37-d63790aedde5 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:36.181301 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 65/134] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:45:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 09:45:36.186721 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e946ae-b3e6-4ea9-8eb8-0680239d803b None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:36.187300 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e946ae-b3e6-4ea9-8eb8-0680239d803b None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:36.187791 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e946ae-b3e6-4ea9-8eb8-0680239d803b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:36.209985 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-01e946ae-b3e6-4ea9-8eb8-0680239d803b None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:45:36.211675 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-01e946ae-b3e6-4ea9-8eb8-0680239d803b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:45:36.214613 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:36.233297 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e034ab5-9708-4118-a3c2-02327c9342d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:36.233937 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e034ab5-9708-4118-a3c2-02327c9342d0 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:36.234234 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e034ab5-9708-4118-a3c2-02327c9342d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:36.234517 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e034ab5-9708-4118-a3c2-02327c9342d0 None admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:36.235321 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e034ab5-9708-4118-a3c2-02327c9342d0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:45:36.235639 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e034ab5-9708-4118-a3c2-02327c9342d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:36.237254 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e034ab5-9708-4118-a3c2-02327c9342d0 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:36.253269 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:36.265723 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 70/135] 10.222.0.64 () {56 vars in 1101 bytes} [Tue Aug 2 09:45:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 385 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:45:36.296106 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e034ab5-9708-4118-a3c2-02327c9342d0 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:36.307229 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 66/136] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:45:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 09:45:36.334307 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e946ae-b3e6-4ea9-8eb8-0680239d803b None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:36.442060 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e034ab5-9708-4118-a3c2-02327c9342d0 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:36.453574 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e946ae-b3e6-4ea9-8eb8-0680239d803b None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:36.504660 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68bd57a-b6fa-444a-abbd-ff8621a48cd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:36.505652 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4056afe7-34a6-4e5e-ac3a-a89e92830c75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:36.506585 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68bd57a-b6fa-444a-abbd-ff8621a48cd4 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:36.507439 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4056afe7-34a6-4e5e-ac3a-a89e92830c75 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:36.507943 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68bd57a-b6fa-444a-abbd-ff8621a48cd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:36.508453 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4056afe7-34a6-4e5e-ac3a-a89e92830c75 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:36.508941 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68bd57a-b6fa-444a-abbd-ff8621a48cd4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:36.509411 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4056afe7-34a6-4e5e-ac3a-a89e92830c75 None admin] PATH_INFO: `/v3/users/6b35b3486c4b4db6a107718cf2807616` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:36.511523 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4056afe7-34a6-4e5e-ac3a-a89e92830c75 None admin] RBAC: Authorizing `identity:get_user(user_id=6b35b3486c4b4db6a107718cf2807616)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:36.515351 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4056afe7-34a6-4e5e-ac3a-a89e92830c75 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:36.519923 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 71/137] 10.222.0.64 () {56 vars in 1177 bytes} [Tue Aug 2 09:45:36 2022] GET /identity/v3/users/6b35b3486c4b4db6a107718cf2807616 => generated 275 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:45:36.525462 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4056afe7-34a6-4e5e-ac3a-a89e92830c75 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:36.531275 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68bd57a-b6fa-444a-abbd-ff8621a48cd4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 09:45:36.541561 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-c68bd57a-b6fa-444a-abbd-ff8621a48cd4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 02 09:45:36.541561 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:45:36.541561 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:45:36.541561 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:45:36.541561 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:45:36.541561 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:45:36.541561 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:45:36.541561 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:45:36.541561 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:45:36.541561 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:45:36.541561 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:45:36.541561 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:45:36.541561 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 09:45:36.541561 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 09:45:36.541561 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 09:45:36.541561 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 09:45:36.542535 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 09:45:36.542535 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 09:45:36.542535 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 09:45:36.542535 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 09:45:36.542535 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:45:36.542535 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:45:36.542535 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:45:36.542535 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:45:36.542535 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:45:36.542535 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:45:36.542535 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:45:36.542535 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:45:36.542535 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:45:36.542535 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:45:36.542535 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:45:36.542535 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:45:36.543344 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:45:36.543344 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:45:36.543344 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:45:36.543344 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:45:36.543344 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 09:45:36.543344 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 09:45:36.543344 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 09:45:36.543344 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 09:45:36.543344 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 09:45:36.543344 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 09:45:36.543344 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 02 09:45:36.543344 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:45:36.548281 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 67/138] 10.222.0.64 () {56 vars in 1099 bytes} [Tue Aug 2 09:45:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 09:45:36.554313 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68bd57a-b6fa-444a-abbd-ff8621a48cd4 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:36.603339 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:36.640283 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:36.727899 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4056afe7-34a6-4e5e-ac3a-a89e92830c75 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:36.776138 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9d1cc1-3a29-4a9c-9d53-e8a1a248a701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:36.777341 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9d1cc1-3a29-4a9c-9d53-e8a1a248a701 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:36.777843 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9d1cc1-3a29-4a9c-9d53-e8a1a248a701 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:36.778344 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9d1cc1-3a29-4a9c-9d53-e8a1a248a701 None admin] PATH_INFO: `/v3/projects/94a149b343344045872ab5da9199fc56` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:36.779921 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9d1cc1-3a29-4a9c-9d53-e8a1a248a701 None admin] RBAC: Authorizing `identity:get_project(project_id=94a149b343344045872ab5da9199fc56)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:36.781236 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68bd57a-b6fa-444a-abbd-ff8621a48cd4 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:36.782215 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9d1cc1-3a29-4a9c-9d53-e8a1a248a701 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:36.784550 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 72/139] 10.222.0.64 () {56 vars in 1186 bytes} [Tue Aug 2 09:45:36 2022] GET /identity/v3/projects/94a149b343344045872ab5da9199fc56 => generated 312 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:45:36.791674 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9d1cc1-3a29-4a9c-9d53-e8a1a248a701 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:36.834940 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ff02cc-bacd-415b-b409-285050a73acd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:36.835578 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ff02cc-bacd-415b-b409-285050a73acd None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:36.835871 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ff02cc-bacd-415b-b409-285050a73acd None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:36.836154 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ff02cc-bacd-415b-b409-285050a73acd None admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:36.843282 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ff02cc-bacd-415b-b409-285050a73acd None admin] RBAC: Adding query filter params (name=member) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:45:36.843282 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ff02cc-bacd-415b-b409-285050a73acd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:36.843282 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ff02cc-bacd-415b-b409-285050a73acd None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:36.860161 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 68/140] 10.222.0.64 () {56 vars in 1101 bytes} [Tue Aug 2 09:45:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:45:36.866348 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ff02cc-bacd-415b-b409-285050a73acd None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:36.973896 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9d1cc1-3a29-4a9c-9d53-e8a1a248a701 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:37.013804 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:37.027415 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:37.052409 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c596dec-2553-43ad-b666-faec761b259a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:37.053468 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c596dec-2553-43ad-b666-faec761b259a None admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:37.054000 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c596dec-2553-43ad-b666-faec761b259a None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:37.054475 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c596dec-2553-43ad-b666-faec761b259a None admin] PATH_INFO: `/v3/projects/94a149b343344045872ab5da9199fc56/users/6b35b3486c4b4db6a107718cf2807616/roles/66c1074780954f8e87ed11a5c4e8c36a` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:37.055912 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ff02cc-bacd-415b-b409-285050a73acd None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:37.059540 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c596dec-2553-43ad-b666-faec761b259a None admin] RBAC: Authorizing `identity:create_grant(project_id=94a149b343344045872ab5da9199fc56, user_id=6b35b3486c4b4db6a107718cf2807616, role_id=66c1074780954f8e87ed11a5c4e8c36a)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:37.061254 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c596dec-2553-43ad-b666-faec761b259a None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:37.086852 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 73/141] 10.222.0.64 () {58 vars in 1439 bytes} [Tue Aug 2 09:45:36 2022] PUT /identity/v3/projects/94a149b343344045872ab5da9199fc56/users/6b35b3486c4b4db6a107718cf2807616/roles/66c1074780954f8e87ed11a5c4e8c36a => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:45:37.132335 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b6717b-4227-440e-a066-7e353d81a1ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:37.133486 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b6717b-4227-440e-a066-7e353d81a1ee None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:37.133993 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b6717b-4227-440e-a066-7e353d81a1ee None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:37.134495 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b6717b-4227-440e-a066-7e353d81a1ee None admin] PATH_INFO: `/v3/users/6b35b3486c4b4db6a107718cf2807616` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:37.136896 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b6717b-4227-440e-a066-7e353d81a1ee None admin] RBAC: Authorizing `identity:get_user(user_id=6b35b3486c4b4db6a107718cf2807616)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:37.139598 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b6717b-4227-440e-a066-7e353d81a1ee None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:37.150896 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 69/142] 10.222.0.64 () {56 vars in 1177 bytes} [Tue Aug 2 09:45:36 2022] GET /identity/v3/users/6b35b3486c4b4db6a107718cf2807616 => generated 275 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:45:37.156729 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c596dec-2553-43ad-b666-faec761b259a None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:37.275284 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c596dec-2553-43ad-b666-faec761b259a None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:37.312716 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7715b1d8-fb5c-4a82-872d-37d34487a621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:37.313743 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7715b1d8-fb5c-4a82-872d-37d34487a621 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:37.314265 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7715b1d8-fb5c-4a82-872d-37d34487a621 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:37.314755 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7715b1d8-fb5c-4a82-872d-37d34487a621 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:37.378147 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:37.416054 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7715b1d8-fb5c-4a82-872d-37d34487a621 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:37.417599 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7715b1d8-fb5c-4a82-872d-37d34487a621 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:37.533225 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 74/143] 10.222.0.64 () {58 vars in 1282 bytes} [Tue Aug 2 09:45:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 02 09:45:37.538831 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b6717b-4227-440e-a066-7e353d81a1ee None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:37.732765 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b6717b-4227-440e-a066-7e353d81a1ee None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:37.776221 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11965049-54e6-48c6-bad8-27856853dacc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:37.777191 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11965049-54e6-48c6-bad8-27856853dacc None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:37.777758 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11965049-54e6-48c6-bad8-27856853dacc None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:37.778221 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11965049-54e6-48c6-bad8-27856853dacc None admin] PATH_INFO: `/v3/projects/ae78e3e21c0448c0b412e161cc374652` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:37.785129 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11965049-54e6-48c6-bad8-27856853dacc None admin] RBAC: Authorizing `identity:get_project(project_id=ae78e3e21c0448c0b412e161cc374652)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:37.787272 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11965049-54e6-48c6-bad8-27856853dacc None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:37.789211 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 70/144] 10.222.0.64 () {56 vars in 1186 bytes} [Tue Aug 2 09:45:37 2022] GET /identity/v3/projects/ae78e3e21c0448c0b412e161cc374652 => generated 298 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:45:37.795155 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7715b1d8-fb5c-4a82-872d-37d34487a621 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:37.903596 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:37.913594 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700a760b-728a-4c43-8e3e-7c871a29bf77 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:37.913898 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700a760b-728a-4c43-8e3e-7c871a29bf77 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:37.914186 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700a760b-728a-4c43-8e3e-7c871a29bf77 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:37.915561 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 71/145] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:45:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 09:45:37.921370 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ec255c-a59b-452b-b425-f94a79b41e2b None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:37.921663 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ec255c-a59b-452b-b425-f94a79b41e2b None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:37.922011 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ec255c-a59b-452b-b425-f94a79b41e2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:37.955640 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-27ec255c-a59b-452b-b425-f94a79b41e2b None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:45:37.958980 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-27ec255c-a59b-452b-b425-f94a79b41e2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:45:37.969452 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7715b1d8-fb5c-4a82-872d-37d34487a621 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:38.012187 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:38.050054 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dc284b-bac4-4917-bcb0-e40aa75364d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:38.051162 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dc284b-bac4-4917-bcb0-e40aa75364d1 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:38.051449 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dc284b-bac4-4917-bcb0-e40aa75364d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:38.051723 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dc284b-bac4-4917-bcb0-e40aa75364d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:38.052569 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dc284b-bac4-4917-bcb0-e40aa75364d1 None admin] RBAC: Adding query filter params (role.id=66c1074780954f8e87ed11a5c4e8c36a, scope.project.id=ae78e3e21c0448c0b412e161cc374652, user.id=6b35b3486c4b4db6a107718cf2807616) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:45:38.052889 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dc284b-bac4-4917-bcb0-e40aa75364d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:38.056643 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dc284b-bac4-4917-bcb0-e40aa75364d1 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:38.072075 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 75/146] 10.222.0.64 () {56 vars in 1374 bytes} [Tue Aug 2 09:45:37 2022] GET /identity/v3/role_assignments?user.id=6b35b3486c4b4db6a107718cf2807616&scope.project.id=ae78e3e21c0448c0b412e161cc374652&role.id=66c1074780954f8e87ed11a5c4e8c36a => generated 608 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:45:38.096076 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 72/147] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:45:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 09:45:38.151037 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dc284b-bac4-4917-bcb0-e40aa75364d1 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:38.302211 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dc284b-bac4-4917-bcb0-e40aa75364d1 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:38.343763 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0a0322-7b9a-44d0-ad98-3b5e430c9478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:38.344835 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0a0322-7b9a-44d0-ad98-3b5e430c9478 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:38.345326 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0a0322-7b9a-44d0-ad98-3b5e430c9478 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:38.345836 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0a0322-7b9a-44d0-ad98-3b5e430c9478 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:38.347814 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0a0322-7b9a-44d0-ad98-3b5e430c9478 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:38.349534 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0a0322-7b9a-44d0-ad98-3b5e430c9478 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:38.356288 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 76/148] 10.222.0.64 () {56 vars in 1108 bytes} [Tue Aug 2 09:45:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:45:38.358056 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ec255c-a59b-452b-b425-f94a79b41e2b None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:38.471507 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ec255c-a59b-452b-b425-f94a79b41e2b None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:38.513802 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75403f1c-fe15-4412-be8e-0a40667bd79c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:38.514795 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75403f1c-fe15-4412-be8e-0a40667bd79c None admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:38.515297 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75403f1c-fe15-4412-be8e-0a40667bd79c None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:38.515796 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75403f1c-fe15-4412-be8e-0a40667bd79c None admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:38.516733 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75403f1c-fe15-4412-be8e-0a40667bd79c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:38.518401 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75403f1c-fe15-4412-be8e-0a40667bd79c None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:38.558623 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 73/149] 10.222.0.64 () {60 vars in 1132 bytes} [Tue Aug 2 09:45:38 2022] POST /identity/v3/users => generated 283 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:45:38.619853 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7df0941-2042-4abf-8076-e880896f5671 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:38.620515 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7df0941-2042-4abf-8076-e880896f5671 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:38.621005 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7df0941-2042-4abf-8076-e880896f5671 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:38.622694 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 77/150] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:45:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:45:38.628657 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0c040e-0dfb-4b97-9c4d-0b2e507c50d1 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:38.629295 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0c040e-0dfb-4b97-9c4d-0b2e507c50d1 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:38.629825 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0c040e-0dfb-4b97-9c4d-0b2e507c50d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:38.652175 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-9b0c040e-0dfb-4b97-9c4d-0b2e507c50d1 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:45:38.653807 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-9b0c040e-0dfb-4b97-9c4d-0b2e507c50d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:45:38.656336 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:38.748027 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 74/151] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:45:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 09:45:38.781354 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7df0941-2042-4abf-8076-e880896f5671 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:38.807602 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:38.928782 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7df0941-2042-4abf-8076-e880896f5671 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:38.973047 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a0b76b-d889-42d6-a840-db38e8b0f72c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:38.973698 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a0b76b-d889-42d6-a840-db38e8b0f72c None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:38.973986 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a0b76b-d889-42d6-a840-db38e8b0f72c None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:38.974263 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a0b76b-d889-42d6-a840-db38e8b0f72c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:38.986605 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a0b76b-d889-42d6-a840-db38e8b0f72c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 09:45:38.992811 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-71a0b76b-d889-42d6-a840-db38e8b0f72c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 02 09:45:38.992811 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:45:38.992811 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:45:38.992811 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:45:38.992811 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:45:38.992811 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:45:38.992811 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:45:38.992811 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:45:38.992811 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:45:38.992811 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:45:38.992811 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:45:38.992811 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:45:38.992811 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 09:45:38.992811 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 09:45:38.992811 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 09:45:38.992811 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 09:45:38.993658 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 09:45:38.993658 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 09:45:38.993658 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 09:45:38.993658 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 09:45:38.993658 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:45:38.993658 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:45:38.993658 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:45:38.993658 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:45:38.993658 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:45:38.993658 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:45:38.993658 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:45:38.993658 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:45:38.993658 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:45:38.993658 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:45:38.993658 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:45:38.993658 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:45:38.994432 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:45:38.994432 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:45:38.994432 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:45:38.994432 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:45:38.994432 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 09:45:38.994432 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 09:45:38.994432 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 09:45:38.994432 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 09:45:38.994432 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 09:45:38.994432 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 09:45:38.994432 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 02 09:45:38.994432 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:45:38.994432 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 78/152] 10.222.0.64 () {56 vars in 1099 bytes} [Tue Aug 2 09:45:38 2022] GET /identity/v3/roles/reader => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:45:39.001334 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0c040e-0dfb-4b97-9c4d-0b2e507c50d1 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:39.136435 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0c040e-0dfb-4b97-9c4d-0b2e507c50d1 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:39.181362 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54137376-dccf-4330-bf4b-580e535e3e81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:39.182225 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54137376-dccf-4330-bf4b-580e535e3e81 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:39.182665 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54137376-dccf-4330-bf4b-580e535e3e81 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:39.183085 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54137376-dccf-4330-bf4b-580e535e3e81 None admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:39.183951 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54137376-dccf-4330-bf4b-580e535e3e81 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:45:39.184422 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54137376-dccf-4330-bf4b-580e535e3e81 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:39.185989 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54137376-dccf-4330-bf4b-580e535e3e81 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:39.192511 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:39.200192 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 75/153] 10.222.0.64 () {56 vars in 1101 bytes} [Tue Aug 2 09:45:39 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:45:39.206035 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a0b76b-d889-42d6-a840-db38e8b0f72c None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:39.293740 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:39.312552 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8f2068-17cd-4160-8491-a74373a5c4b9 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:39.313010 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8f2068-17cd-4160-8491-a74373a5c4b9 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:39.313442 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8f2068-17cd-4160-8491-a74373a5c4b9 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:39.314983 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 76/154] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:45:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:45:39.326208 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8bb075-2bcc-4d7c-a2e9-9b02dea7be91 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:39.326679 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8bb075-2bcc-4d7c-a2e9-9b02dea7be91 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:39.327114 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8bb075-2bcc-4d7c-a2e9-9b02dea7be91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:39.336060 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a0b76b-d889-42d6-a840-db38e8b0f72c None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:39.369408 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-cc8bb075-2bcc-4d7c-a2e9-9b02dea7be91 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:45:39.370727 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-cc8bb075-2bcc-4d7c-a2e9-9b02dea7be91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:45:39.379416 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6bc0c7-f744-4585-8d99-4e4f43990a76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:39.380187 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6bc0c7-f744-4585-8d99-4e4f43990a76 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:39.380590 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6bc0c7-f744-4585-8d99-4e4f43990a76 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:39.380943 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6bc0c7-f744-4585-8d99-4e4f43990a76 None admin] PATH_INFO: `/v3/users/c7f7673261604df6a51226b783e563b5` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:39.382366 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6bc0c7-f744-4585-8d99-4e4f43990a76 None admin] RBAC: Authorizing `identity:get_user(user_id=c7f7673261604df6a51226b783e563b5)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:39.384034 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6bc0c7-f744-4585-8d99-4e4f43990a76 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:39.386094 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 79/155] 10.222.0.64 () {56 vars in 1177 bytes} [Tue Aug 2 09:45:39 2022] GET /identity/v3/users/c7f7673261604df6a51226b783e563b5 => generated 289 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:45:39.392548 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6bc0c7-f744-4585-8d99-4e4f43990a76 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:39.500216 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 77/156] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:45:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 09:45:39.557982 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6bc0c7-f744-4585-8d99-4e4f43990a76 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:39.563043 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dab2d5c-fe40-42e5-a019-0e0d3d9d66b5 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:39.563338 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dab2d5c-fe40-42e5-a019-0e0d3d9d66b5 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:39.563617 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dab2d5c-fe40-42e5-a019-0e0d3d9d66b5 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:39.565399 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 78/157] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:45:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:45:39.566673 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dab2d5c-fe40-42e5-a019-0e0d3d9d66b5 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:39.616048 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a2de37-530d-4502-b4f7-15b6f35ea1e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:39.616747 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a2de37-530d-4502-b4f7-15b6f35ea1e4 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:39.617027 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a2de37-530d-4502-b4f7-15b6f35ea1e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:39.617385 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a2de37-530d-4502-b4f7-15b6f35ea1e4 None admin] PATH_INFO: `/v3/projects/ae78e3e21c0448c0b412e161cc374652` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:39.618505 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a2de37-530d-4502-b4f7-15b6f35ea1e4 None admin] RBAC: Authorizing `identity:get_project(project_id=ae78e3e21c0448c0b412e161cc374652)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:39.620141 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a2de37-530d-4502-b4f7-15b6f35ea1e4 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:39.622062 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 80/158] 10.222.0.64 () {56 vars in 1186 bytes} [Tue Aug 2 09:45:39 2022] GET /identity/v3/projects/ae78e3e21c0448c0b412e161cc374652 => generated 298 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:45:39.623045 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21d41ba-6e63-41d6-b714-4ca9cca77686 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:39.623333 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21d41ba-6e63-41d6-b714-4ca9cca77686 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:39.623610 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21d41ba-6e63-41d6-b714-4ca9cca77686 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:39.652827 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-e21d41ba-6e63-41d6-b714-4ca9cca77686 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:45:39.654035 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-e21d41ba-6e63-41d6-b714-4ca9cca77686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:45:39.656176 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:39.717692 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dab2d5c-fe40-42e5-a019-0e0d3d9d66b5 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:39.747493 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 81/159] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:45:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 09:45:39.748348 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21d41ba-6e63-41d6-b714-4ca9cca77686 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:39.776759 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:39.793170 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c694a3-f6ac-4d07-9be4-0ef1d855e0c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:39.793701 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c694a3-f6ac-4d07-9be4-0ef1d855e0c0 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:39.793988 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c694a3-f6ac-4d07-9be4-0ef1d855e0c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:39.794268 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c694a3-f6ac-4d07-9be4-0ef1d855e0c0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:39.823916 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c694a3-f6ac-4d07-9be4-0ef1d855e0c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 09:45:39.829408 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-12c694a3-f6ac-4d07-9be4-0ef1d855e0c0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 09:45:39.829408 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:45:39.829408 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:45:39.829408 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:45:39.829408 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:45:39.829408 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:45:39.829408 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:45:39.829408 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:45:39.829408 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:45:39.829408 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:45:39.829408 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:45:39.829408 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:45:39.829408 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 09:45:39.829408 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 09:45:39.829408 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 09:45:39.829408 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 09:45:39.830294 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 09:45:39.830294 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 09:45:39.830294 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 09:45:39.830294 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 09:45:39.830294 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:45:39.830294 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:45:39.830294 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:45:39.830294 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:45:39.830294 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:45:39.830294 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:45:39.830294 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:45:39.830294 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:45:39.830294 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:45:39.830294 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:45:39.830294 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:45:39.830294 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:45:39.831095 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:45:39.831095 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:45:39.831095 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:45:39.831095 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:45:39.831095 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 09:45:39.831095 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 09:45:39.831095 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 09:45:39.831095 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 09:45:39.831095 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 09:45:39.831095 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 09:45:39.831095 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 09:45:39.831095 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:45:39.831095 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 79/160] 10.222.0.64 () {56 vars in 1114 bytes} [Tue Aug 2 09:45:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:45:39.832291 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c694a3-f6ac-4d07-9be4-0ef1d855e0c0 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:39.915205 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21d41ba-6e63-41d6-b714-4ca9cca77686 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:39.977903 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba9d208-424b-4714-8d84-eb82d701d4f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:39.978954 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba9d208-424b-4714-8d84-eb82d701d4f5 None admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:39.979438 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba9d208-424b-4714-8d84-eb82d701d4f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:39.979941 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba9d208-424b-4714-8d84-eb82d701d4f5 None admin] PATH_INFO: `/v3/projects/ae78e3e21c0448c0b412e161cc374652/users/c7f7673261604df6a51226b783e563b5/roles/7fce0045dfa049bdb7ae0d2424de9f32` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:39.985900 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba9d208-424b-4714-8d84-eb82d701d4f5 None admin] RBAC: Authorizing `identity:create_grant(project_id=ae78e3e21c0448c0b412e161cc374652, user_id=c7f7673261604df6a51226b783e563b5, role_id=7fce0045dfa049bdb7ae0d2424de9f32)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:39.988315 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba9d208-424b-4714-8d84-eb82d701d4f5 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:39.996862 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c694a3-f6ac-4d07-9be4-0ef1d855e0c0 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:40.002963 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:40.010555 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 82/161] 10.222.0.64 () {58 vars in 1439 bytes} [Tue Aug 2 09:45:39 2022] PUT /identity/v3/projects/ae78e3e21c0448c0b412e161cc374652/users/c7f7673261604df6a51226b783e563b5/roles/7fce0045dfa049bdb7ae0d2424de9f32 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:45:40.013028 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba9d208-424b-4714-8d84-eb82d701d4f5 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:40.068890 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c5389b-e72c-4ae6-98ef-3dc39bf48c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:40.070537 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c5389b-e72c-4ae6-98ef-3dc39bf48c1d None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:40.071183 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c5389b-e72c-4ae6-98ef-3dc39bf48c1d None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:40.071845 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c5389b-e72c-4ae6-98ef-3dc39bf48c1d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:40.094812 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c5389b-e72c-4ae6-98ef-3dc39bf48c1d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 09:45:40.106287 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-67c5389b-e72c-4ae6-98ef-3dc39bf48c1d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 09:45:40.106287 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:45:40.106287 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:45:40.106287 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:45:40.106287 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:45:40.106287 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:45:40.106287 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:45:40.106287 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:45:40.106287 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:45:40.106287 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:45:40.106287 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:45:40.106287 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:45:40.106287 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 09:45:40.106287 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 09:45:40.106287 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 09:45:40.106287 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 09:45:40.107246 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 09:45:40.107246 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 09:45:40.107246 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 09:45:40.107246 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 09:45:40.107246 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:45:40.107246 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:45:40.107246 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:45:40.107246 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:45:40.107246 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:45:40.107246 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:45:40.107246 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:45:40.107246 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:45:40.107246 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:45:40.107246 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:45:40.107246 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:45:40.107246 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:45:40.108108 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:45:40.108108 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:45:40.108108 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:45:40.108108 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:45:40.108108 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 09:45:40.108108 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 09:45:40.108108 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 09:45:40.108108 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 09:45:40.108108 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 09:45:40.108108 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 09:45:40.108108 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 09:45:40.108108 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:45:40.111072 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 80/162] 10.222.0.64 () {56 vars in 1096 bytes} [Tue Aug 2 09:45:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:45:40.116788 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c5389b-e72c-4ae6-98ef-3dc39bf48c1d None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:40.192296 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba9d208-424b-4714-8d84-eb82d701d4f5 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:40.204278 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:40.254106 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beab6ea8-f8a0-48fb-b97d-b6f1b2153ff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:40.254863 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beab6ea8-f8a0-48fb-b97d-b6f1b2153ff3 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:40.255250 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beab6ea8-f8a0-48fb-b97d-b6f1b2153ff3 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:40.255645 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beab6ea8-f8a0-48fb-b97d-b6f1b2153ff3 None admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:40.256413 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beab6ea8-f8a0-48fb-b97d-b6f1b2153ff3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:45:40.256791 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beab6ea8-f8a0-48fb-b97d-b6f1b2153ff3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:40.258375 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beab6ea8-f8a0-48fb-b97d-b6f1b2153ff3 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:40.279112 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 83/163] 10.222.0.64 () {56 vars in 1111 bytes} [Tue Aug 2 09:45:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:45:40.282251 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beab6ea8-f8a0-48fb-b97d-b6f1b2153ff3 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:40.289209 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c5389b-e72c-4ae6-98ef-3dc39bf48c1d None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:40.334011 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632243cc-f72e-4553-a906-0163035f4ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:40.334942 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632243cc-f72e-4553-a906-0163035f4ae6 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:40.335406 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632243cc-f72e-4553-a906-0163035f4ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:40.335859 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632243cc-f72e-4553-a906-0163035f4ae6 None admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:40.336841 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632243cc-f72e-4553-a906-0163035f4ae6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:45:40.337329 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632243cc-f72e-4553-a906-0163035f4ae6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:40.339045 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632243cc-f72e-4553-a906-0163035f4ae6 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:40.349345 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 81/164] 10.222.0.64 () {56 vars in 1099 bytes} [Tue Aug 2 09:45:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:45:40.351182 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e358d3-ed94-4190-bfa9-a5431bd524bc None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:40.352187 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e358d3-ed94-4190-bfa9-a5431bd524bc None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:40.352746 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e358d3-ed94-4190-bfa9-a5431bd524bc None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:40.354452 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 82/165] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:45:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:45:40.355363 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e358d3-ed94-4190-bfa9-a5431bd524bc None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:40.364576 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:40.408842 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beab6ea8-f8a0-48fb-b97d-b6f1b2153ff3 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:40.455292 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769963e6-6890-4d76-8428-e237ceb71548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:40.455946 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769963e6-6890-4d76-8428-e237ceb71548 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:40.456273 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769963e6-6890-4d76-8428-e237ceb71548 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:40.457069 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769963e6-6890-4d76-8428-e237ceb71548 None admin] PATH_INFO: `/v3/users/6b35b3486c4b4db6a107718cf2807616` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:40.458917 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769963e6-6890-4d76-8428-e237ceb71548 None admin] RBAC: Authorizing `identity:get_user(user_id=6b35b3486c4b4db6a107718cf2807616)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:40.460424 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769963e6-6890-4d76-8428-e237ceb71548 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:40.462195 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 84/166] 10.222.0.64 () {56 vars in 1177 bytes} [Tue Aug 2 09:45:40 2022] GET /identity/v3/users/6b35b3486c4b4db6a107718cf2807616 => generated 275 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:45:40.463147 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3888a5dd-d11d-4961-a835-a01c2ad5c668 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:40.463431 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3888a5dd-d11d-4961-a835-a01c2ad5c668 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:40.463708 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3888a5dd-d11d-4961-a835-a01c2ad5c668 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:40.505617 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e358d3-ed94-4190-bfa9-a5431bd524bc None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:40.512904 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-3888a5dd-d11d-4961-a835-a01c2ad5c668 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:45:40.514159 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-3888a5dd-d11d-4961-a835-a01c2ad5c668 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:45:40.528134 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:40.570725 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1f99b9-6f4a-453a-b4ab-0fda082b4cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:40.571387 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1f99b9-6f4a-453a-b4ab-0fda082b4cba None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:40.571676 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1f99b9-6f4a-453a-b4ab-0fda082b4cba None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:40.571957 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1f99b9-6f4a-453a-b4ab-0fda082b4cba None admin] PATH_INFO: `/v3/users/5c5f9105c00b4d00a6249c76315d5e3e` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:40.573583 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1f99b9-6f4a-453a-b4ab-0fda082b4cba None admin] RBAC: Authorizing `identity:get_user(user_id=5c5f9105c00b4d00a6249c76315d5e3e)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:40.575362 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1f99b9-6f4a-453a-b4ab-0fda082b4cba None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:40.577540 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 83/167] 10.222.0.64 () {56 vars in 1177 bytes} [Tue Aug 2 09:45:40 2022] GET /identity/v3/users/5c5f9105c00b4d00a6249c76315d5e3e => generated 247 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:45:40.577831 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1f99b9-6f4a-453a-b4ab-0fda082b4cba None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:40.645846 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 85/168] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:45:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 09:45:40.647028 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3888a5dd-d11d-4961-a835-a01c2ad5c668 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:40.767201 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1f99b9-6f4a-453a-b4ab-0fda082b4cba None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:40.773672 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:40.812940 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f8fac5-8d6a-4331-80a5-36fdeb14c819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:40.813983 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f8fac5-8d6a-4331-80a5-36fdeb14c819 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:40.814473 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f8fac5-8d6a-4331-80a5-36fdeb14c819 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:40.814978 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f8fac5-8d6a-4331-80a5-36fdeb14c819 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:40.841807 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3888a5dd-d11d-4961-a835-a01c2ad5c668 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:40.873217 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:40.897739 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f8fac5-8d6a-4331-80a5-36fdeb14c819 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:40.899386 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f8fac5-8d6a-4331-80a5-36fdeb14c819 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:40.917727 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7740f7-537a-452e-85ab-fa2d1e44fcef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:40.918370 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7740f7-537a-452e-85ab-fa2d1e44fcef None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:40.918658 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7740f7-537a-452e-85ab-fa2d1e44fcef None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:40.918932 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7740f7-537a-452e-85ab-fa2d1e44fcef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:41.011890 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 84/169] 10.222.0.64 () {58 vars in 1282 bytes} [Tue Aug 2 09:45:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 09:45:41.012847 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f8fac5-8d6a-4331-80a5-36fdeb14c819 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:41.061392 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7740f7-537a-452e-85ab-fa2d1e44fcef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:41.062906 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7740f7-537a-452e-85ab-fa2d1e44fcef None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:41.113619 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:41.136123 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f8fac5-8d6a-4331-80a5-36fdeb14c819 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:41.170655 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:41.184997 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe3cd96-fa29-4813-a8c8-f179084cc2ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:41.185947 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe3cd96-fa29-4813-a8c8-f179084cc2ee None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:41.186439 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe3cd96-fa29-4813-a8c8-f179084cc2ee None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:41.186869 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe3cd96-fa29-4813-a8c8-f179084cc2ee None admin] PATH_INFO: `/v3/roles/member` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:41.202262 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe3cd96-fa29-4813-a8c8-f179084cc2ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 09:45:41.214480 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-0fe3cd96-fa29-4813-a8c8-f179084cc2ee None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 02 09:45:41.214480 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:45:41.214480 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:45:41.214480 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:45:41.214480 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:45:41.214480 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:45:41.214480 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:45:41.214480 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:45:41.214480 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:45:41.214480 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:45:41.214480 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:45:41.214480 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:45:41.214480 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 09:45:41.214480 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 09:45:41.214480 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 09:45:41.214480 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 09:45:41.215478 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 09:45:41.215478 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 09:45:41.215478 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 09:45:41.215478 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 09:45:41.215478 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:45:41.215478 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:45:41.215478 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:45:41.215478 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:45:41.215478 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:45:41.215478 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:45:41.215478 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:45:41.215478 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:45:41.215478 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:45:41.215478 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:45:41.215478 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:45:41.215478 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:45:41.216322 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:45:41.216322 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:45:41.216322 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:45:41.216322 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:45:41.216322 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 09:45:41.216322 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 09:45:41.216322 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 09:45:41.216322 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 09:45:41.216322 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 09:45:41.216322 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 09:45:41.216322 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 02 09:45:41.216322 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:45:41.220371 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 85/170] 10.222.0.64 () {56 vars in 1099 bytes} [Tue Aug 2 09:45:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:45:41.221315 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe3cd96-fa29-4813-a8c8-f179084cc2ee None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:41.247213 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 86/171] 10.222.0.64 () {58 vars in 1282 bytes} [Tue Aug 2 09:45:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 600 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 09:45:41.248092 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7740f7-537a-452e-85ab-fa2d1e44fcef None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:41.422791 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7740f7-537a-452e-85ab-fa2d1e44fcef None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:41.473433 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894e529c-16e0-4119-b02b-035908182894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:41.474047 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894e529c-16e0-4119-b02b-035908182894 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:41.474329 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894e529c-16e0-4119-b02b-035908182894 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:41.474602 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894e529c-16e0-4119-b02b-035908182894 None admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:41.475331 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894e529c-16e0-4119-b02b-035908182894 None admin] RBAC: Adding query filter params (name=member) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:45:41.475641 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894e529c-16e0-4119-b02b-035908182894 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:41.477309 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894e529c-16e0-4119-b02b-035908182894 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:41.485880 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 87/172] 10.222.0.64 () {56 vars in 1101 bytes} [Tue Aug 2 09:45:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:45:41.486521 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894e529c-16e0-4119-b02b-035908182894 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:41.509603 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:41.542153 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe3cd96-fa29-4813-a8c8-f179084cc2ee None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:41.620459 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62321e59-f804-42b7-b4b7-84121a7f2c3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:41.621110 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62321e59-f804-42b7-b4b7-84121a7f2c3d None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:41.621993 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62321e59-f804-42b7-b4b7-84121a7f2c3d None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:41.622466 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62321e59-f804-42b7-b4b7-84121a7f2c3d None admin] PATH_INFO: `/v3/projects/ae78e3e21c0448c0b412e161cc374652` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:41.624045 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62321e59-f804-42b7-b4b7-84121a7f2c3d None admin] RBAC: Authorizing `identity:get_project(project_id=ae78e3e21c0448c0b412e161cc374652)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:41.625550 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62321e59-f804-42b7-b4b7-84121a7f2c3d None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:41.627218 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 86/173] 10.222.0.64 () {56 vars in 1186 bytes} [Tue Aug 2 09:45:41 2022] GET /identity/v3/projects/ae78e3e21c0448c0b412e161cc374652 => generated 298 bytes in 406 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:45:41.627987 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62321e59-f804-42b7-b4b7-84121a7f2c3d None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:41.634213 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894e529c-16e0-4119-b02b-035908182894 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:41.650840 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:41.673393 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc218b41-2108-4326-ba77-890cebb16159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:41.674270 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc218b41-2108-4326-ba77-890cebb16159 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:41.674700 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc218b41-2108-4326-ba77-890cebb16159 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:41.675183 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc218b41-2108-4326-ba77-890cebb16159 None admin] PATH_INFO: `/v3/projects/ae78e3e21c0448c0b412e161cc374652` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:41.677152 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc218b41-2108-4326-ba77-890cebb16159 None admin] RBAC: Authorizing `identity:get_project(project_id=ae78e3e21c0448c0b412e161cc374652)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:41.678752 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc218b41-2108-4326-ba77-890cebb16159 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:41.680904 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 88/174] 10.222.0.64 () {56 vars in 1186 bytes} [Tue Aug 2 09:45:41 2022] GET /identity/v3/projects/ae78e3e21c0448c0b412e161cc374652 => generated 298 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:45:41.681986 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc218b41-2108-4326-ba77-890cebb16159 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:41.780272 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:41.798375 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc218b41-2108-4326-ba77-890cebb16159 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:41.859853 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69d0fcb-213b-4152-99ea-0da19794f523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:41.860537 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69d0fcb-213b-4152-99ea-0da19794f523 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:41.860829 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69d0fcb-213b-4152-99ea-0da19794f523 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:41.861112 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69d0fcb-213b-4152-99ea-0da19794f523 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:41.861934 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69d0fcb-213b-4152-99ea-0da19794f523 None admin] RBAC: Adding query filter params (role.id=06dfd237063c42a4bff1ad8ca29c07b0, scope.project.id=ae78e3e21c0448c0b412e161cc374652, user.id=6b35b3486c4b4db6a107718cf2807616) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:45:41.862255 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69d0fcb-213b-4152-99ea-0da19794f523 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:41.864315 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69d0fcb-213b-4152-99ea-0da19794f523 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:41.876984 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 89/175] 10.222.0.64 () {56 vars in 1374 bytes} [Tue Aug 2 09:45:41 2022] GET /identity/v3/role_assignments?user.id=6b35b3486c4b4db6a107718cf2807616&scope.project.id=ae78e3e21c0448c0b412e161cc374652&role.id=06dfd237063c42a4bff1ad8ca29c07b0 => generated 608 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:45:41.877813 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69d0fcb-213b-4152-99ea-0da19794f523 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:41.984296 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62321e59-f804-42b7-b4b7-84121a7f2c3d None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:42.082801 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baec5af-f52f-46f0-9f79-b992570f0760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:42.083811 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baec5af-f52f-46f0-9f79-b992570f0760 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:42.084276 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baec5af-f52f-46f0-9f79-b992570f0760 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:42.084723 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baec5af-f52f-46f0-9f79-b992570f0760 None admin] PATH_INFO: `/v3/users/6b35b3486c4b4db6a107718cf2807616` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:42.089525 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4baec5af-f52f-46f0-9f79-b992570f0760 None admin] RBAC: Authorizing `identity:get_user(user_id=6b35b3486c4b4db6a107718cf2807616)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:42.092049 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4baec5af-f52f-46f0-9f79-b992570f0760 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:42.096676 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 87/176] 10.222.0.64 () {56 vars in 1177 bytes} [Tue Aug 2 09:45:41 2022] GET /identity/v3/users/6b35b3486c4b4db6a107718cf2807616 => generated 275 bytes in 467 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:45:42.104647 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baec5af-f52f-46f0-9f79-b992570f0760 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:42.108873 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69d0fcb-213b-4152-99ea-0da19794f523 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:42.187971 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5d336d-ae69-498c-9d4f-d786df451d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:42.188703 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5d336d-ae69-498c-9d4f-d786df451d4f None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:42.188992 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5d336d-ae69-498c-9d4f-d786df451d4f None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:42.189325 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5d336d-ae69-498c-9d4f-d786df451d4f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:42.190214 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5d336d-ae69-498c-9d4f-d786df451d4f None admin] RBAC: Adding query filter params (role.id=1000cf318fa14915b8091599eaead5de, scope.project.id=ae78e3e21c0448c0b412e161cc374652, user.id=5c5f9105c00b4d00a6249c76315d5e3e) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:45:42.190480 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5d336d-ae69-498c-9d4f-d786df451d4f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:42.196224 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5d336d-ae69-498c-9d4f-d786df451d4f None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:42.217999 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 90/177] 10.222.0.64 () {56 vars in 1374 bytes} [Tue Aug 2 09:45:41 2022] GET /identity/v3/role_assignments?user.id=5c5f9105c00b4d00a6249c76315d5e3e&scope.project.id=ae78e3e21c0448c0b412e161cc374652&role.id=1000cf318fa14915b8091599eaead5de => generated 608 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:45:42.236817 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:42.256661 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baec5af-f52f-46f0-9f79-b992570f0760 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:42.320604 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73abd8b4-c711-4903-95e4-65cdbff939c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:42.321260 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73abd8b4-c711-4903-95e4-65cdbff939c0 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:42.321544 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73abd8b4-c711-4903-95e4-65cdbff939c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:42.321818 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73abd8b4-c711-4903-95e4-65cdbff939c0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:42.430750 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73abd8b4-c711-4903-95e4-65cdbff939c0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:42.432318 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73abd8b4-c711-4903-95e4-65cdbff939c0 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:42.529486 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:42.545867 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3649633e-e241-46a4-8276-4d72edc72196 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:42.546175 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3649633e-e241-46a4-8276-4d72edc72196 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:42.546437 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3649633e-e241-46a4-8276-4d72edc72196 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:42.547830 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 91/178] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:45:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:45:42.556803 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 88/179] 10.222.0.64 () {58 vars in 1282 bytes} [Tue Aug 2 09:45:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 09:45:42.563477 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73abd8b4-c711-4903-95e4-65cdbff939c0 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:42.563925 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4043f5-3924-4a6b-aca5-d9f4e706760b None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:42.564222 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4043f5-3924-4a6b-aca5-d9f4e706760b None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:42.564603 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4043f5-3924-4a6b-aca5-d9f4e706760b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:42.601909 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-1f4043f5-3924-4a6b-aca5-d9f4e706760b None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:45:42.604085 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-1f4043f5-3924-4a6b-aca5-d9f4e706760b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:45:42.624208 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:42.713836 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 92/180] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:45:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 09:45:42.741776 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4043f5-3924-4a6b-aca5-d9f4e706760b None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:42.746687 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73abd8b4-c711-4903-95e4-65cdbff939c0 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:42.816850 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa32f23-1308-4255-b4a3-43e29e03dca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:42.817878 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa32f23-1308-4255-b4a3-43e29e03dca2 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:42.818370 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa32f23-1308-4255-b4a3-43e29e03dca2 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:42.818838 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa32f23-1308-4255-b4a3-43e29e03dca2 None admin] PATH_INFO: `/v3/projects/94a149b343344045872ab5da9199fc56` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:42.820314 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa32f23-1308-4255-b4a3-43e29e03dca2 None admin] RBAC: Authorizing `identity:get_project(project_id=94a149b343344045872ab5da9199fc56)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:42.821981 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa32f23-1308-4255-b4a3-43e29e03dca2 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:42.823882 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 89/181] 10.222.0.64 () {56 vars in 1186 bytes} [Tue Aug 2 09:45:42 2022] GET /identity/v3/projects/94a149b343344045872ab5da9199fc56 => generated 312 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:45:42.825404 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b94290c-f13f-4b12-be5c-80e6a50996b3 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:42.825901 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b94290c-f13f-4b12-be5c-80e6a50996b3 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:42.826387 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b94290c-f13f-4b12-be5c-80e6a50996b3 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:42.827850 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 90/182] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:45:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:45:42.832822 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b94290c-f13f-4b12-be5c-80e6a50996b3 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:42.857085 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4043f5-3924-4a6b-aca5-d9f4e706760b None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:42.912518 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:42.912902 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d02e4b2-5968-4ff5-9f71-e7e58650e13a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:42.913868 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d02e4b2-5968-4ff5-9f71-e7e58650e13a None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:42.914402 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d02e4b2-5968-4ff5-9f71-e7e58650e13a None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:42.914727 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d02e4b2-5968-4ff5-9f71-e7e58650e13a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:42.926447 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d02e4b2-5968-4ff5-9f71-e7e58650e13a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 09:45:42.931573 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-4d02e4b2-5968-4ff5-9f71-e7e58650e13a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 09:45:42.931573 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:45:42.931573 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:45:42.931573 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:45:42.931573 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:45:42.931573 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:45:42.931573 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:45:42.931573 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:45:42.931573 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:45:42.931573 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:45:42.931573 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:45:42.931573 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:45:42.931573 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 09:45:42.931573 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 09:45:42.931573 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 09:45:42.931573 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 09:45:42.932540 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 09:45:42.932540 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 09:45:42.932540 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 09:45:42.932540 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 09:45:42.932540 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:45:42.932540 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:45:42.932540 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:45:42.932540 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:45:42.932540 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:45:42.932540 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:45:42.932540 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:45:42.932540 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:45:42.932540 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:45:42.932540 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:45:42.932540 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:45:42.932540 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:45:42.933355 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:45:42.933355 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:45:42.933355 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:45:42.933355 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:45:42.933355 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 09:45:42.933355 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 09:45:42.933355 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 09:45:42.933355 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 09:45:42.933355 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 09:45:42.933355 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 09:45:42.933355 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 09:45:42.933355 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:45:42.933355 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 93/183] 10.222.0.64 () {56 vars in 1096 bytes} [Tue Aug 2 09:45:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 09:45:42.934563 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad969a30-f0b7-4ed6-9cc6-07719f4506a0 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:42.934563 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad969a30-f0b7-4ed6-9cc6-07719f4506a0 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:42.934716 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad969a30-f0b7-4ed6-9cc6-07719f4506a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:42.950590 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:42.957908 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-ad969a30-f0b7-4ed6-9cc6-07719f4506a0 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:45:42.959182 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-ad969a30-f0b7-4ed6-9cc6-07719f4506a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:45:42.995363 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b94290c-f13f-4b12-be5c-80e6a50996b3 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:43.036871 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4831fdcb-56a3-4072-908c-6a8baf94bf07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:43.037454 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 94/184] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:45:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 09:45:43.037562 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4831fdcb-56a3-4072-908c-6a8baf94bf07 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:43.037857 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4831fdcb-56a3-4072-908c-6a8baf94bf07 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:43.038810 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad969a30-f0b7-4ed6-9cc6-07719f4506a0 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:43.044643 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4831fdcb-56a3-4072-908c-6a8baf94bf07 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:43.046827 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4831fdcb-56a3-4072-908c-6a8baf94bf07 None admin] RBAC: Adding query filter params (role.id=66c1074780954f8e87ed11a5c4e8c36a, scope.project.id=94a149b343344045872ab5da9199fc56, user.id=6b35b3486c4b4db6a107718cf2807616) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:45:43.052707 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4831fdcb-56a3-4072-908c-6a8baf94bf07 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:43.054537 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4831fdcb-56a3-4072-908c-6a8baf94bf07 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:43.064602 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 91/185] 10.222.0.64 () {56 vars in 1374 bytes} [Tue Aug 2 09:45:42 2022] GET /identity/v3/role_assignments?user.id=6b35b3486c4b4db6a107718cf2807616&scope.project.id=94a149b343344045872ab5da9199fc56&role.id=66c1074780954f8e87ed11a5c4e8c36a => generated 608 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:45:43.067836 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4831fdcb-56a3-4072-908c-6a8baf94bf07 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:43.177693 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad969a30-f0b7-4ed6-9cc6-07719f4506a0 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:43.216920 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3937fb-fb75-4ba1-b36c-c990a498fc93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:43.217643 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3937fb-fb75-4ba1-b36c-c990a498fc93 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:43.217747 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4831fdcb-56a3-4072-908c-6a8baf94bf07 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:43.217936 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3937fb-fb75-4ba1-b36c-c990a498fc93 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:43.218219 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3937fb-fb75-4ba1-b36c-c990a498fc93 None admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:43.219012 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3937fb-fb75-4ba1-b36c-c990a498fc93 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:45:43.219337 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3937fb-fb75-4ba1-b36c-c990a498fc93 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:43.220804 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3937fb-fb75-4ba1-b36c-c990a498fc93 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:43.223777 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:43.228645 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:43.238440 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 95/186] 10.222.0.64 () {56 vars in 1099 bytes} [Tue Aug 2 09:45:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:45:43.238440 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af79017-b1b3-4813-b934-1c97b08b9e0a None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:43.238440 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af79017-b1b3-4813-b934-1c97b08b9e0a None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:43.238440 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af79017-b1b3-4813-b934-1c97b08b9e0a None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:43.238440 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 96/187] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:45:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:45:43.242558 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de64e0ef-9341-4775-87da-86a7d9d78ec0 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:43.242856 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de64e0ef-9341-4775-87da-86a7d9d78ec0 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:43.243137 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de64e0ef-9341-4775-87da-86a7d9d78ec0 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:43.244535 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 97/188] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:45:43 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:45:43.245249 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de64e0ef-9341-4775-87da-86a7d9d78ec0 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:43.268027 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4436cb4c-0111-432c-ba2e-9727474b59c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:43.268909 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4436cb4c-0111-432c-ba2e-9727474b59c1 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:43.269764 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4436cb4c-0111-432c-ba2e-9727474b59c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:43.270478 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4436cb4c-0111-432c-ba2e-9727474b59c1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:43.271338 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4436cb4c-0111-432c-ba2e-9727474b59c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:43.272858 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4436cb4c-0111-432c-ba2e-9727474b59c1 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:43.274635 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 92/189] 10.222.0.64 () {56 vars in 1108 bytes} [Tue Aug 2 09:45:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:45:43.279236 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf037449-e2c4-4192-bf20-2c7098f66ce3 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:43.279236 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf037449-e2c4-4192-bf20-2c7098f66ce3 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:43.279236 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf037449-e2c4-4192-bf20-2c7098f66ce3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:43.295345 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-bf037449-e2c4-4192-bf20-2c7098f66ce3 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:45:43.296604 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-bf037449-e2c4-4192-bf20-2c7098f66ce3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:45:43.370073 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de64e0ef-9341-4775-87da-86a7d9d78ec0 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:43.373179 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 93/190] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:45:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 09:45:43.374431 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294fa14c-5ed8-4d6b-b36c-8ad9e5f94924 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:43.374724 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294fa14c-5ed8-4d6b-b36c-8ad9e5f94924 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:43.375000 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294fa14c-5ed8-4d6b-b36c-8ad9e5f94924 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:43.393044 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-294fa14c-5ed8-4d6b-b36c-8ad9e5f94924 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:45:43.394790 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-294fa14c-5ed8-4d6b-b36c-8ad9e5f94924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:45:43.412324 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8169e421-f78c-4603-902a-9fec9259e92c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:43.412981 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8169e421-f78c-4603-902a-9fec9259e92c None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:43.413266 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8169e421-f78c-4603-902a-9fec9259e92c None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:43.413583 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8169e421-f78c-4603-902a-9fec9259e92c None admin] PATH_INFO: `/v3/users/3a33da6a369a4c3b9f5a663e28f6d306` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:43.427432 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8169e421-f78c-4603-902a-9fec9259e92c None admin] RBAC: Authorizing `identity:get_user(user_id=3a33da6a369a4c3b9f5a663e28f6d306)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:43.428929 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8169e421-f78c-4603-902a-9fec9259e92c None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:43.430586 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 98/191] 10.222.0.64 () {56 vars in 1177 bytes} [Tue Aug 2 09:45:43 2022] GET /identity/v3/users/3a33da6a369a4c3b9f5a663e28f6d306 => generated 283 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:45:43.431338 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8169e421-f78c-4603-902a-9fec9259e92c None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:43.467934 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 94/192] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:45:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 09:45:43.468902 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294fa14c-5ed8-4d6b-b36c-8ad9e5f94924 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:43.502153 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:43.505111 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:43.546608 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8169e421-f78c-4603-902a-9fec9259e92c None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:43.569334 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294fa14c-5ed8-4d6b-b36c-8ad9e5f94924 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:43.580616 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538d2484-d09f-47b4-88d7-086dd8eb4631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:43.581512 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538d2484-d09f-47b4-88d7-086dd8eb4631 None admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:43.581811 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538d2484-d09f-47b4-88d7-086dd8eb4631 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:43.582089 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538d2484-d09f-47b4-88d7-086dd8eb4631 None admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:43.582772 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538d2484-d09f-47b4-88d7-086dd8eb4631 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:43.584208 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538d2484-d09f-47b4-88d7-086dd8eb4631 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:43.603929 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80429c56-3749-45d6-922e-72e09e044f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:43.604609 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80429c56-3749-45d6-922e-72e09e044f78 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:43.604897 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80429c56-3749-45d6-922e-72e09e044f78 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:43.605119 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 99/193] 10.222.0.64 () {60 vars in 1132 bytes} [Tue Aug 2 09:45:43 2022] POST /identity/v3/users => generated 297 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:45:43.605245 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80429c56-3749-45d6-922e-72e09e044f78 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:43.605940 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538d2484-d09f-47b4-88d7-086dd8eb4631 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:43.631285 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80429c56-3749-45d6-922e-72e09e044f78 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 09:45:43.636915 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-80429c56-3749-45d6-922e-72e09e044f78 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 09:45:43.636915 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:45:43.636915 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:45:43.636915 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:45:43.636915 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:45:43.636915 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:45:43.636915 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:45:43.636915 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:45:43.636915 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:45:43.636915 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:45:43.636915 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:45:43.636915 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:45:43.636915 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 09:45:43.636915 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 09:45:43.636915 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 09:45:43.636915 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 09:45:43.637942 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 09:45:43.637942 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 09:45:43.637942 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 09:45:43.637942 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 09:45:43.637942 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:45:43.637942 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:45:43.637942 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:45:43.637942 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:45:43.637942 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:45:43.637942 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:45:43.637942 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:45:43.637942 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:45:43.637942 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:45:43.637942 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:45:43.637942 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:45:43.637942 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:45:43.638750 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:45:43.638750 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:45:43.638750 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:45:43.638750 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:45:43.638750 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 09:45:43.638750 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 09:45:43.638750 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 09:45:43.638750 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 09:45:43.638750 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 09:45:43.638750 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 09:45:43.638750 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 09:45:43.638750 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:45:43.638750 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 95/194] 10.222.0.64 () {56 vars in 1114 bytes} [Tue Aug 2 09:45:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:45:43.639405 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80429c56-3749-45d6-922e-72e09e044f78 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:43.727511 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538d2484-d09f-47b4-88d7-086dd8eb4631 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:43.743163 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:43.747769 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:43.756883 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80429c56-3749-45d6-922e-72e09e044f78 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:43.770132 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6023d5-2fd9-4685-a953-e75e97e3942b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:43.770763 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6023d5-2fd9-4685-a953-e75e97e3942b None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:43.771055 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6023d5-2fd9-4685-a953-e75e97e3942b None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:43.771334 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6023d5-2fd9-4685-a953-e75e97e3942b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:43.803062 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f257d88-5866-4a4c-b1a8-4cecc919cf84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:43.803732 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f257d88-5866-4a4c-b1a8-4cecc919cf84 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:43.803999 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f257d88-5866-4a4c-b1a8-4cecc919cf84 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:43.804287 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f257d88-5866-4a4c-b1a8-4cecc919cf84 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:43.820067 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f257d88-5866-4a4c-b1a8-4cecc919cf84 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 09:45:43.825602 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-7f257d88-5866-4a4c-b1a8-4cecc919cf84 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 02 09:45:43.825602 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:45:43.825602 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:45:43.825602 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:45:43.825602 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:45:43.825602 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:45:43.825602 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:45:43.825602 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:45:43.825602 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:45:43.825602 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:45:43.825602 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:45:43.825602 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:45:43.825602 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 09:45:43.825602 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 09:45:43.825602 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 09:45:43.825602 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 09:45:43.826510 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 09:45:43.826510 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 09:45:43.826510 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 09:45:43.826510 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 09:45:43.826510 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:45:43.826510 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:45:43.826510 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:45:43.826510 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:45:43.826510 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:45:43.826510 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:45:43.826510 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:45:43.826510 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:45:43.826510 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:45:43.826510 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:45:43.826510 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:45:43.826510 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:45:43.827306 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:45:43.827306 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:45:43.827306 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:45:43.827306 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:45:43.827306 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 09:45:43.827306 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 09:45:43.827306 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 09:45:43.827306 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 09:45:43.827306 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 09:45:43.827306 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 09:45:43.827306 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 02 09:45:43.827306 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:45:43.827306 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 96/195] 10.222.0.64 () {56 vars in 1099 bytes} [Tue Aug 2 09:45:43 2022] GET /identity/v3/roles/reader => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:45:43.827306 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f257d88-5866-4a4c-b1a8-4cecc919cf84 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:43.861959 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6023d5-2fd9-4685-a953-e75e97e3942b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:43.863347 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6023d5-2fd9-4685-a953-e75e97e3942b None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:43.960868 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f257d88-5866-4a4c-b1a8-4cecc919cf84 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:43.997589 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 100/196] 10.222.0.64 () {58 vars in 1282 bytes} [Tue Aug 2 09:45:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 09:45:43.998539 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6023d5-2fd9-4685-a953-e75e97e3942b None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:44.014407 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9614c73d-1e7a-4d4d-a226-ec767271acdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:44.015043 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9614c73d-1e7a-4d4d-a226-ec767271acdc None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:44.015294 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:44.015402 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9614c73d-1e7a-4d4d-a226-ec767271acdc None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:44.015579 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9614c73d-1e7a-4d4d-a226-ec767271acdc None admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:44.016387 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9614c73d-1e7a-4d4d-a226-ec767271acdc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:45:44.016676 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9614c73d-1e7a-4d4d-a226-ec767271acdc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:44.018234 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9614c73d-1e7a-4d4d-a226-ec767271acdc None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:44.027145 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 97/197] 10.222.0.64 () {56 vars in 1111 bytes} [Tue Aug 2 09:45:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:45:44.031666 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9614c73d-1e7a-4d4d-a226-ec767271acdc None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:44.047872 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:44.131655 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6023d5-2fd9-4685-a953-e75e97e3942b None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:44.136470 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9614c73d-1e7a-4d4d-a226-ec767271acdc None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:44.171652 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43e8066-c1ea-446e-a8c2-ba14e9dd7d45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:44.172171 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43e8066-c1ea-446e-a8c2-ba14e9dd7d45 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:44.172487 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43e8066-c1ea-446e-a8c2-ba14e9dd7d45 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:44.172766 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43e8066-c1ea-446e-a8c2-ba14e9dd7d45 None admin] PATH_INFO: `/v3/projects/bc7fa79dde7e42ae92a2a42495efdcc2` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:44.173953 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43e8066-c1ea-446e-a8c2-ba14e9dd7d45 None admin] RBAC: Authorizing `identity:get_project(project_id=bc7fa79dde7e42ae92a2a42495efdcc2)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:44.175653 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43e8066-c1ea-446e-a8c2-ba14e9dd7d45 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:44.177187 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 98/198] 10.222.0.64 () {56 vars in 1186 bytes} [Tue Aug 2 09:45:44 2022] GET /identity/v3/projects/bc7fa79dde7e42ae92a2a42495efdcc2 => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:45:44.177963 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43e8066-c1ea-446e-a8c2-ba14e9dd7d45 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:44.202808 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5b3167-03d9-4c5b-bc21-89cc85ce0319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:44.205081 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5b3167-03d9-4c5b-bc21-89cc85ce0319 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:44.206005 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5b3167-03d9-4c5b-bc21-89cc85ce0319 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:44.207409 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5b3167-03d9-4c5b-bc21-89cc85ce0319 None admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:44.209217 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5b3167-03d9-4c5b-bc21-89cc85ce0319 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:45:44.210246 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5b3167-03d9-4c5b-bc21-89cc85ce0319 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:44.213491 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5b3167-03d9-4c5b-bc21-89cc85ce0319 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:44.231604 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 101/199] 10.222.0.64 () {56 vars in 1101 bytes} [Tue Aug 2 09:45:43 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:45:44.233718 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5b3167-03d9-4c5b-bc21-89cc85ce0319 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:44.277157 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:44.293823 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43e8066-c1ea-446e-a8c2-ba14e9dd7d45 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:44.330477 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cc49e5-6b14-4bc3-ad8c-276f61acd1ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:44.331647 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cc49e5-6b14-4bc3-ad8c-276f61acd1ad None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:44.331940 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cc49e5-6b14-4bc3-ad8c-276f61acd1ad None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:44.332220 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cc49e5-6b14-4bc3-ad8c-276f61acd1ad None admin] PATH_INFO: `/v3/users/3a33da6a369a4c3b9f5a663e28f6d306` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:44.334039 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cc49e5-6b14-4bc3-ad8c-276f61acd1ad None admin] RBAC: Authorizing `identity:get_user(user_id=3a33da6a369a4c3b9f5a663e28f6d306)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:44.335554 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cc49e5-6b14-4bc3-ad8c-276f61acd1ad None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:44.337522 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 99/200] 10.222.0.64 () {56 vars in 1177 bytes} [Tue Aug 2 09:45:44 2022] GET /identity/v3/users/3a33da6a369a4c3b9f5a663e28f6d306 => generated 283 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:45:44.339101 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cc49e5-6b14-4bc3-ad8c-276f61acd1ad None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:44.432633 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5b3167-03d9-4c5b-bc21-89cc85ce0319 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:44.454619 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cc49e5-6b14-4bc3-ad8c-276f61acd1ad None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:44.494292 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736ef70e-dad1-4910-bd18-f8bcba840951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:44.494943 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736ef70e-dad1-4910-bd18-f8bcba840951 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:44.495241 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736ef70e-dad1-4910-bd18-f8bcba840951 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:44.495544 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736ef70e-dad1-4910-bd18-f8bcba840951 None admin] PATH_INFO: `/v3/users/c7f7673261604df6a51226b783e563b5` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:44.496864 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736ef70e-dad1-4910-bd18-f8bcba840951 None admin] RBAC: Authorizing `identity:get_user(user_id=c7f7673261604df6a51226b783e563b5)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:44.497327 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663db7f3-9e41-4983-b10f-dc643f4d05c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:44.498329 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736ef70e-dad1-4910-bd18-f8bcba840951 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:44.498955 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663db7f3-9e41-4983-b10f-dc643f4d05c6 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:44.499730 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663db7f3-9e41-4983-b10f-dc643f4d05c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:44.500361 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 100/201] 10.222.0.64 () {56 vars in 1177 bytes} [Tue Aug 2 09:45:44 2022] GET /identity/v3/users/c7f7673261604df6a51226b783e563b5 => generated 289 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:45:44.501400 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663db7f3-9e41-4983-b10f-dc643f4d05c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:44.503292 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663db7f3-9e41-4983-b10f-dc643f4d05c6 None admin] RBAC: Adding query filter params (role.id=1000cf318fa14915b8091599eaead5de, scope.project.id=bc7fa79dde7e42ae92a2a42495efdcc2, user.id=3a33da6a369a4c3b9f5a663e28f6d306) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:45:44.505100 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736ef70e-dad1-4910-bd18-f8bcba840951 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:44.508446 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:44.508993 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663db7f3-9e41-4983-b10f-dc643f4d05c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:44.510848 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663db7f3-9e41-4983-b10f-dc643f4d05c6 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:44.517517 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 102/202] 10.222.0.64 () {56 vars in 1374 bytes} [Tue Aug 2 09:45:44 2022] GET /identity/v3/role_assignments?user.id=3a33da6a369a4c3b9f5a663e28f6d306&scope.project.id=bc7fa79dde7e42ae92a2a42495efdcc2&role.id=1000cf318fa14915b8091599eaead5de => generated 259 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:45:44.519339 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663db7f3-9e41-4983-b10f-dc643f4d05c6 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:44.559497 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:44.682150 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663db7f3-9e41-4983-b10f-dc643f4d05c6 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:44.730744 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736ef70e-dad1-4910-bd18-f8bcba840951 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:44.744994 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4801b2f5-1ed8-4cd6-82d3-f4509607ebe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:44.745782 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4801b2f5-1ed8-4cd6-82d3-f4509607ebe5 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:44.746378 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4801b2f5-1ed8-4cd6-82d3-f4509607ebe5 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:44.746740 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4801b2f5-1ed8-4cd6-82d3-f4509607ebe5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:44.758712 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:44.775320 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebf90d0-275a-42b6-af68-ae500e509092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:44.776226 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebf90d0-275a-42b6-af68-ae500e509092 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:44.776744 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebf90d0-275a-42b6-af68-ae500e509092 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:44.777211 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebf90d0-275a-42b6-af68-ae500e509092 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:44.833158 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4801b2f5-1ed8-4cd6-82d3-f4509607ebe5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:44.835119 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4801b2f5-1ed8-4cd6-82d3-f4509607ebe5 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:44.953307 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebf90d0-275a-42b6-af68-ae500e509092 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:44.954910 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebf90d0-275a-42b6-af68-ae500e509092 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:44.956085 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 103/203] 10.222.0.64 () {58 vars in 1282 bytes} [Tue Aug 2 09:45:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 09:45:44.963272 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4801b2f5-1ed8-4cd6-82d3-f4509607ebe5 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:45.007373 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:45.011043 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:45.087803 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4801b2f5-1ed8-4cd6-82d3-f4509607ebe5 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:45.128736 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b9eb1d-4d65-43a5-85bf-faa92d630594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:45.129650 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b9eb1d-4d65-43a5-85bf-faa92d630594 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:45.130940 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b9eb1d-4d65-43a5-85bf-faa92d630594 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:45.131382 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b9eb1d-4d65-43a5-85bf-faa92d630594 None admin] PATH_INFO: `/v3/projects/ae78e3e21c0448c0b412e161cc374652` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:45.132827 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b9eb1d-4d65-43a5-85bf-faa92d630594 None admin] RBAC: Authorizing `identity:get_project(project_id=ae78e3e21c0448c0b412e161cc374652)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:45.134470 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b9eb1d-4d65-43a5-85bf-faa92d630594 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:45.136312 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 104/204] 10.222.0.64 () {56 vars in 1186 bytes} [Tue Aug 2 09:45:44 2022] GET /identity/v3/projects/ae78e3e21c0448c0b412e161cc374652 => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:45:45.142703 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b9eb1d-4d65-43a5-85bf-faa92d630594 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:45.239841 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 101/205] 10.222.0.64 () {58 vars in 1282 bytes} [Tue Aug 2 09:45:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 739 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 09:45:45.247982 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebf90d0-275a-42b6-af68-ae500e509092 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:45.270691 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b9eb1d-4d65-43a5-85bf-faa92d630594 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:45.280551 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:45.333643 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4017d75-c972-46dd-b8f9-633b2f5c2b68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:45.334272 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4017d75-c972-46dd-b8f9-633b2f5c2b68 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:45.334558 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4017d75-c972-46dd-b8f9-633b2f5c2b68 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:45.334881 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4017d75-c972-46dd-b8f9-633b2f5c2b68 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:45.335677 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4017d75-c972-46dd-b8f9-633b2f5c2b68 None admin] RBAC: Adding query filter params (role.id=7fce0045dfa049bdb7ae0d2424de9f32, scope.project.id=ae78e3e21c0448c0b412e161cc374652, user.id=c7f7673261604df6a51226b783e563b5) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:45:45.335995 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4017d75-c972-46dd-b8f9-633b2f5c2b68 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:45.338689 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4017d75-c972-46dd-b8f9-633b2f5c2b68 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:45.355926 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 105/206] 10.222.0.64 () {56 vars in 1374 bytes} [Tue Aug 2 09:45:45 2022] GET /identity/v3/role_assignments?user.id=c7f7673261604df6a51226b783e563b5&scope.project.id=ae78e3e21c0448c0b412e161cc374652&role.id=7fce0045dfa049bdb7ae0d2424de9f32 => generated 608 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:45:45.408997 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebf90d0-275a-42b6-af68-ae500e509092 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:45.439196 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:45.463260 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd455a9d-53fe-4193-9d3f-91f80883886d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:45.463945 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd455a9d-53fe-4193-9d3f-91f80883886d None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:45.464237 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd455a9d-53fe-4193-9d3f-91f80883886d None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:45.464543 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd455a9d-53fe-4193-9d3f-91f80883886d None admin] PATH_INFO: `/v3/projects/bc7fa79dde7e42ae92a2a42495efdcc2` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:45.465828 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd455a9d-53fe-4193-9d3f-91f80883886d None admin] RBAC: Authorizing `identity:get_project(project_id=bc7fa79dde7e42ae92a2a42495efdcc2)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:45.467320 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd455a9d-53fe-4193-9d3f-91f80883886d None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:45.469273 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 102/207] 10.222.0.64 () {56 vars in 1186 bytes} [Tue Aug 2 09:45:45 2022] GET /identity/v3/projects/bc7fa79dde7e42ae92a2a42495efdcc2 => generated 302 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:45:45.477667 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4017d75-c972-46dd-b8f9-633b2f5c2b68 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:45.611673 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4017d75-c972-46dd-b8f9-633b2f5c2b68 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:45.646456 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47085c8a-7c14-4ccf-9ca2-a48a90cb7b32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:45.647073 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47085c8a-7c14-4ccf-9ca2-a48a90cb7b32 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:45.647355 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47085c8a-7c14-4ccf-9ca2-a48a90cb7b32 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:45.647626 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47085c8a-7c14-4ccf-9ca2-a48a90cb7b32 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:45.648456 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47085c8a-7c14-4ccf-9ca2-a48a90cb7b32 None admin] RBAC: Adding query filter params (role.id=06dfd237063c42a4bff1ad8ca29c07b0, scope.project.id=bc7fa79dde7e42ae92a2a42495efdcc2, user.id=3a33da6a369a4c3b9f5a663e28f6d306) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:45:45.648768 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47085c8a-7c14-4ccf-9ca2-a48a90cb7b32 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:45.650207 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47085c8a-7c14-4ccf-9ca2-a48a90cb7b32 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:45.656318 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 106/208] 10.222.0.64 () {56 vars in 1374 bytes} [Tue Aug 2 09:45:45 2022] GET /identity/v3/role_assignments?user.id=3a33da6a369a4c3b9f5a663e28f6d306&scope.project.id=bc7fa79dde7e42ae92a2a42495efdcc2&role.id=06dfd237063c42a4bff1ad8ca29c07b0 => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:45:45.956557 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214eec2a-00c0-42d6-9036-fc7c7879c2db None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:45.956892 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214eec2a-00c0-42d6-9036-fc7c7879c2db None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:45.957126 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214eec2a-00c0-42d6-9036-fc7c7879c2db None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:45.959566 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 103/209] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:45:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:45:45.966369 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16447346-0987-466a-ab3c-7d9688ade17b None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:45.966700 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16447346-0987-466a-ab3c-7d9688ade17b None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:45.966938 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16447346-0987-466a-ab3c-7d9688ade17b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:45.986924 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-16447346-0987-466a-ab3c-7d9688ade17b None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:45:45.988809 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-16447346-0987-466a-ab3c-7d9688ade17b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:45:46.018292 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:46.067597 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 107/210] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:45:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 09:45:46.081935 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2064e7a-0823-49cf-9b69-c9fc47bbcf9f None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:46.082224 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2064e7a-0823-49cf-9b69-c9fc47bbcf9f None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:46.082505 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2064e7a-0823-49cf-9b69-c9fc47bbcf9f None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:46.083867 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 104/211] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:45:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:45:46.089103 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b1ce22-499e-4c74-93ec-1727e0091df4 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:46.089686 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b1ce22-499e-4c74-93ec-1727e0091df4 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:46.090151 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2064e7a-0823-49cf-9b69-c9fc47bbcf9f None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:46.090351 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b1ce22-499e-4c74-93ec-1727e0091df4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:46.116323 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-56b1ce22-499e-4c74-93ec-1727e0091df4 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:45:46.118429 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-56b1ce22-499e-4c74-93ec-1727e0091df4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:45:46.211173 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2064e7a-0823-49cf-9b69-c9fc47bbcf9f None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:46.211881 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 108/212] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:45:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 09:45:46.235793 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b1ce22-499e-4c74-93ec-1727e0091df4 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:46.253523 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a6a1f1-cc61-4f56-a33b-74bcf510d9c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:46.254528 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a6a1f1-cc61-4f56-a33b-74bcf510d9c6 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:46.255123 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a6a1f1-cc61-4f56-a33b-74bcf510d9c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:46.255595 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a6a1f1-cc61-4f56-a33b-74bcf510d9c6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:46.267363 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a6a1f1-cc61-4f56-a33b-74bcf510d9c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 09:45:46.273828 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-44a6a1f1-cc61-4f56-a33b-74bcf510d9c6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 02 09:45:46.273828 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:45:46.273828 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:45:46.273828 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:45:46.273828 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:45:46.273828 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:45:46.273828 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:45:46.273828 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:45:46.273828 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:45:46.273828 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:45:46.273828 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:45:46.273828 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:45:46.273828 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 09:45:46.273828 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 09:45:46.273828 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 09:45:46.273828 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 09:45:46.274707 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 09:45:46.274707 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 09:45:46.274707 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 09:45:46.274707 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 09:45:46.274707 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:45:46.274707 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:45:46.274707 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:45:46.274707 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:45:46.274707 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:45:46.274707 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:45:46.274707 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:45:46.274707 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:45:46.274707 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:45:46.274707 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:45:46.274707 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:45:46.274707 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:45:46.275531 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:45:46.275531 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:45:46.275531 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:45:46.275531 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:45:46.275531 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 09:45:46.275531 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 09:45:46.275531 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 09:45:46.275531 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 09:45:46.275531 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 09:45:46.275531 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 09:45:46.275531 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 02 09:45:46.275531 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:45:46.278315 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 105/213] 10.222.0.64 () {56 vars in 1099 bytes} [Tue Aug 2 09:45:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 09:45:46.287288 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a6a1f1-cc61-4f56-a33b-74bcf510d9c6 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:46.348031 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:46.355111 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:46.365718 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b1ce22-499e-4c74-93ec-1727e0091df4 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:46.404502 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5898df77-0bb1-4c3f-9148-946a33b7be19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:46.405662 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5898df77-0bb1-4c3f-9148-946a33b7be19 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:46.406306 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5898df77-0bb1-4c3f-9148-946a33b7be19 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:46.406788 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5898df77-0bb1-4c3f-9148-946a33b7be19 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:46.408382 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5898df77-0bb1-4c3f-9148-946a33b7be19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:46.410568 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5898df77-0bb1-4c3f-9148-946a33b7be19 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:46.411592 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a6a1f1-cc61-4f56-a33b-74bcf510d9c6 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:46.417903 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 109/214] 10.222.0.64 () {56 vars in 1108 bytes} [Tue Aug 2 09:45:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:45:46.419499 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5898df77-0bb1-4c3f-9148-946a33b7be19 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:46.457813 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d14ca76-5d28-49e4-bc84-016040533782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:46.458430 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d14ca76-5d28-49e4-bc84-016040533782 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:46.458717 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d14ca76-5d28-49e4-bc84-016040533782 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:46.458992 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d14ca76-5d28-49e4-bc84-016040533782 None admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:46.459720 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d14ca76-5d28-49e4-bc84-016040533782 None admin] RBAC: Adding query filter params (name=member) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:45:46.460028 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d14ca76-5d28-49e4-bc84-016040533782 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:46.462553 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d14ca76-5d28-49e4-bc84-016040533782 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:46.472771 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 106/215] 10.222.0.64 () {56 vars in 1101 bytes} [Tue Aug 2 09:45:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:45:46.478210 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d14ca76-5d28-49e4-bc84-016040533782 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:46.537612 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5898df77-0bb1-4c3f-9148-946a33b7be19 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:46.575894 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917cb943-4e2f-410f-a18e-1cb9895801cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:46.577055 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917cb943-4e2f-410f-a18e-1cb9895801cf None admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:46.577548 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917cb943-4e2f-410f-a18e-1cb9895801cf None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:46.578029 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917cb943-4e2f-410f-a18e-1cb9895801cf None admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:46.579084 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917cb943-4e2f-410f-a18e-1cb9895801cf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:46.580906 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917cb943-4e2f-410f-a18e-1cb9895801cf None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:46.604311 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d14ca76-5d28-49e4-bc84-016040533782 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:46.610555 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 110/216] 10.222.0.64 () {60 vars in 1132 bytes} [Tue Aug 2 09:45:46 2022] POST /identity/v3/users => generated 297 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 09:45:46.627042 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:46.652959 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908a8d86-52d7-4ba4-9955-227c17d06401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:46.653715 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908a8d86-52d7-4ba4-9955-227c17d06401 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:46.654011 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908a8d86-52d7-4ba4-9955-227c17d06401 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:46.654284 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908a8d86-52d7-4ba4-9955-227c17d06401 None admin] PATH_INFO: `/v3/users/31178481748442ada86cab1c8bc75b18` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:46.670985 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908a8d86-52d7-4ba4-9955-227c17d06401 None admin] RBAC: Authorizing `identity:get_user(user_id=31178481748442ada86cab1c8bc75b18)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:46.672532 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908a8d86-52d7-4ba4-9955-227c17d06401 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:46.674652 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 107/217] 10.222.0.64 () {56 vars in 1177 bytes} [Tue Aug 2 09:45:46 2022] GET /identity/v3/users/31178481748442ada86cab1c8bc75b18 => generated 297 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:45:46.681470 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917cb943-4e2f-410f-a18e-1cb9895801cf None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:46.706353 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:46.799599 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917cb943-4e2f-410f-a18e-1cb9895801cf None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:46.839232 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29eff3cf-997e-429e-88a8-a371c41fcd2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:46.839954 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eff3cf-997e-429e-88a8-a371c41fcd2f None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:46.840282 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eff3cf-997e-429e-88a8-a371c41fcd2f None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:46.840606 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eff3cf-997e-429e-88a8-a371c41fcd2f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:46.914314 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29eff3cf-997e-429e-88a8-a371c41fcd2f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:46.915520 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb2a36d-4b47-4f83-81e5-ad52120e0ec0 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:46.915920 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb2a36d-4b47-4f83-81e5-ad52120e0ec0 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:46.916375 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb2a36d-4b47-4f83-81e5-ad52120e0ec0 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:46.916950 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29eff3cf-997e-429e-88a8-a371c41fcd2f None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:46.918250 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 108/218] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:45:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 09:45:46.923171 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56026ab-a063-471f-b613-2fdb4383b6d4 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:46.923466 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56026ab-a063-471f-b613-2fdb4383b6d4 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:46.923748 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56026ab-a063-471f-b613-2fdb4383b6d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:46.944620 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:46.953816 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-f56026ab-a063-471f-b613-2fdb4383b6d4 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:45:46.955591 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-f56026ab-a063-471f-b613-2fdb4383b6d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:45:47.045065 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 109/219] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:45:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 09:45:47.049489 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 111/220] 10.222.0.64 () {58 vars in 1282 bytes} [Tue Aug 2 09:45:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 09:45:47.051144 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56026ab-a063-471f-b613-2fdb4383b6d4 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:47.067476 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29eff3cf-997e-429e-88a8-a371c41fcd2f None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:47.169951 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56026ab-a063-471f-b613-2fdb4383b6d4 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:47.174327 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:47.194741 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29eff3cf-997e-429e-88a8-a371c41fcd2f None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:47.214652 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:47.219045 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96410b54-04e1-4431-9c90-bf9883976b83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:47.220176 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96410b54-04e1-4431-9c90-bf9883976b83 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:47.220636 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96410b54-04e1-4431-9c90-bf9883976b83 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:47.221388 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96410b54-04e1-4431-9c90-bf9883976b83 None admin] PATH_INFO: `/v3/projects/bc7fa79dde7e42ae92a2a42495efdcc2` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:47.224633 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96410b54-04e1-4431-9c90-bf9883976b83 None admin] RBAC: Authorizing `identity:get_project(project_id=bc7fa79dde7e42ae92a2a42495efdcc2)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:47.227442 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96410b54-04e1-4431-9c90-bf9883976b83 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:47.230338 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 110/221] 10.222.0.64 () {56 vars in 1186 bytes} [Tue Aug 2 09:45:47 2022] GET /identity/v3/projects/bc7fa79dde7e42ae92a2a42495efdcc2 => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:45:47.236069 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96410b54-04e1-4431-9c90-bf9883976b83 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:47.238750 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b587fb16-d974-4e58-8489-faa8e3c81da2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:47.239404 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b587fb16-d974-4e58-8489-faa8e3c81da2 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:47.239690 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b587fb16-d974-4e58-8489-faa8e3c81da2 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:47.240021 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b587fb16-d974-4e58-8489-faa8e3c81da2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:47.252575 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b587fb16-d974-4e58-8489-faa8e3c81da2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 09:45:47.257913 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-b587fb16-d974-4e58-8489-faa8e3c81da2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 09:45:47.257913 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:45:47.257913 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:45:47.257913 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:45:47.257913 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:45:47.257913 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:45:47.257913 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:45:47.257913 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:45:47.257913 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:45:47.257913 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:45:47.257913 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:45:47.257913 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:45:47.257913 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 09:45:47.257913 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 09:45:47.257913 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 09:45:47.257913 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 09:45:47.259031 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 09:45:47.259031 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 09:45:47.259031 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 09:45:47.259031 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 09:45:47.259031 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:45:47.259031 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:45:47.259031 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:45:47.259031 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:45:47.259031 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:45:47.259031 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:45:47.259031 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:45:47.259031 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:45:47.259031 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:45:47.259031 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:45:47.259031 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:45:47.259031 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:45:47.260045 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:45:47.260045 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:45:47.260045 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:45:47.260045 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:45:47.260045 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 09:45:47.260045 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 09:45:47.260045 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 09:45:47.260045 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 09:45:47.260045 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 09:45:47.260045 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 09:45:47.260045 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 09:45:47.260045 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:45:47.263985 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 112/222] 10.222.0.64 () {56 vars in 1096 bytes} [Tue Aug 2 09:45:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:45:47.265252 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b587fb16-d974-4e58-8489-faa8e3c81da2 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:47.361806 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96410b54-04e1-4431-9c90-bf9883976b83 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:47.402632 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db1ac55-8232-4640-8b69-f6751393031b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:47.403756 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db1ac55-8232-4640-8b69-f6751393031b None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:47.404320 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db1ac55-8232-4640-8b69-f6751393031b None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:47.405005 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db1ac55-8232-4640-8b69-f6751393031b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:47.406913 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db1ac55-8232-4640-8b69-f6751393031b None admin] RBAC: Adding query filter params (role.id=66c1074780954f8e87ed11a5c4e8c36a, scope.project.id=bc7fa79dde7e42ae92a2a42495efdcc2, user.id=31178481748442ada86cab1c8bc75b18) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:45:47.407385 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db1ac55-8232-4640-8b69-f6751393031b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:47.409076 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db1ac55-8232-4640-8b69-f6751393031b None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:47.416311 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 111/223] 10.222.0.64 () {56 vars in 1374 bytes} [Tue Aug 2 09:45:47 2022] GET /identity/v3/role_assignments?user.id=31178481748442ada86cab1c8bc75b18&scope.project.id=bc7fa79dde7e42ae92a2a42495efdcc2&role.id=66c1074780954f8e87ed11a5c4e8c36a => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:45:47.480959 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b587fb16-d974-4e58-8489-faa8e3c81da2 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:47.535293 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d3a73e-bced-42cc-9b10-05b8c760cc08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:47.535947 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d3a73e-bced-42cc-9b10-05b8c760cc08 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:47.536235 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d3a73e-bced-42cc-9b10-05b8c760cc08 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:47.536536 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d3a73e-bced-42cc-9b10-05b8c760cc08 None admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:47.537294 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d3a73e-bced-42cc-9b10-05b8c760cc08 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:45:47.537606 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d3a73e-bced-42cc-9b10-05b8c760cc08 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:47.539045 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d3a73e-bced-42cc-9b10-05b8c760cc08 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:47.551365 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 113/224] 10.222.0.64 () {56 vars in 1099 bytes} [Tue Aug 2 09:45:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:45:47.557299 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db1ac55-8232-4640-8b69-f6751393031b None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:47.605257 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:47.685862 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db1ac55-8232-4640-8b69-f6751393031b None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:47.731970 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f6b237-566a-4c34-825f-869947690e78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:47.736819 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f6b237-566a-4c34-825f-869947690e78 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:47.737119 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f6b237-566a-4c34-825f-869947690e78 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:47.737465 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f6b237-566a-4c34-825f-869947690e78 None admin] PATH_INFO: `/v3/users/3a33da6a369a4c3b9f5a663e28f6d306` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:47.738833 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f6b237-566a-4c34-825f-869947690e78 None admin] RBAC: Authorizing `identity:get_user(user_id=3a33da6a369a4c3b9f5a663e28f6d306)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:47.740360 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f6b237-566a-4c34-825f-869947690e78 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:47.742055 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 112/225] 10.222.0.64 () {56 vars in 1177 bytes} [Tue Aug 2 09:45:47 2022] GET /identity/v3/users/3a33da6a369a4c3b9f5a663e28f6d306 => generated 283 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:45:47.748028 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d3a73e-bced-42cc-9b10-05b8c760cc08 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:47.827299 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:47.832058 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9d0914-b956-4155-93a3-ac55045568c2 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:47.832836 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9d0914-b956-4155-93a3-ac55045568c2 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:47.833326 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9d0914-b956-4155-93a3-ac55045568c2 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:47.835938 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 113/226] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:45:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:45:47.841133 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df80fe6-62d4-4313-8f5e-a01668b20fc4 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:47.841426 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df80fe6-62d4-4313-8f5e-a01668b20fc4 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:47.841712 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df80fe6-62d4-4313-8f5e-a01668b20fc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:47.882497 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-4df80fe6-62d4-4313-8f5e-a01668b20fc4 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:45:47.886802 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-4df80fe6-62d4-4313-8f5e-a01668b20fc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:45:47.944799 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d3a73e-bced-42cc-9b10-05b8c760cc08 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:48.015627 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5b23db-a234-4c6c-8fab-e31651410fd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:48.017700 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5b23db-a234-4c6c-8fab-e31651410fd4 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:48.018588 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5b23db-a234-4c6c-8fab-e31651410fd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:48.019496 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5b23db-a234-4c6c-8fab-e31651410fd4 None admin] PATH_INFO: `/v3/projects/bc7fa79dde7e42ae92a2a42495efdcc2` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:48.021386 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 114/227] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:45:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 09:45:48.022593 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5b23db-a234-4c6c-8fab-e31651410fd4 None admin] RBAC: Authorizing `identity:get_project(project_id=bc7fa79dde7e42ae92a2a42495efdcc2)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:48.024415 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5b23db-a234-4c6c-8fab-e31651410fd4 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:48.029839 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 114/228] 10.222.0.64 () {56 vars in 1186 bytes} [Tue Aug 2 09:45:47 2022] GET /identity/v3/projects/bc7fa79dde7e42ae92a2a42495efdcc2 => generated 302 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:45:48.032498 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df80fe6-62d4-4313-8f5e-a01668b20fc4 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:48.049040 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5b23db-a234-4c6c-8fab-e31651410fd4 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:48.090290 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:48.233530 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df80fe6-62d4-4313-8f5e-a01668b20fc4 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:48.261234 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5b23db-a234-4c6c-8fab-e31651410fd4 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:48.277003 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:48.311780 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b921c019-b2e0-4c51-9717-345589111a59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:48.312507 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b921c019-b2e0-4c51-9717-345589111a59 None admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:48.312797 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b921c019-b2e0-4c51-9717-345589111a59 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:48.313076 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b921c019-b2e0-4c51-9717-345589111a59 None admin] PATH_INFO: `/v3/projects/bc7fa79dde7e42ae92a2a42495efdcc2/users/3a33da6a369a4c3b9f5a663e28f6d306/roles/1000cf318fa14915b8091599eaead5de` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:48.316760 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b921c019-b2e0-4c51-9717-345589111a59 None admin] RBAC: Authorizing `identity:create_grant(project_id=bc7fa79dde7e42ae92a2a42495efdcc2, user_id=3a33da6a369a4c3b9f5a663e28f6d306, role_id=1000cf318fa14915b8091599eaead5de)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:48.318231 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b921c019-b2e0-4c51-9717-345589111a59 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:48.334035 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf5a5f1-c0bc-44f1-b251-b8780458fc46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:48.334780 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf5a5f1-c0bc-44f1-b251-b8780458fc46 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:48.335072 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf5a5f1-c0bc-44f1-b251-b8780458fc46 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:48.335356 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf5a5f1-c0bc-44f1-b251-b8780458fc46 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:48.340816 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 115/229] 10.222.0.64 () {58 vars in 1439 bytes} [Tue Aug 2 09:45:48 2022] PUT /identity/v3/projects/bc7fa79dde7e42ae92a2a42495efdcc2/users/3a33da6a369a4c3b9f5a663e28f6d306/roles/1000cf318fa14915b8091599eaead5de => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:45:48.365510 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf5a5f1-c0bc-44f1-b251-b8780458fc46 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 09:45:48.371258 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-ecf5a5f1-c0bc-44f1-b251-b8780458fc46 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 09:45:48.371258 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:45:48.371258 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:45:48.371258 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:45:48.371258 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:45:48.371258 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:45:48.371258 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:45:48.371258 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:45:48.371258 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:45:48.371258 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:45:48.371258 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:45:48.371258 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:45:48.371258 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 09:45:48.371258 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 09:45:48.371258 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 09:45:48.371258 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 09:45:48.372272 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 09:45:48.372272 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 09:45:48.372272 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 09:45:48.372272 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 09:45:48.372272 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:45:48.372272 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:45:48.372272 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:45:48.372272 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:45:48.372272 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:45:48.372272 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:45:48.372272 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:45:48.372272 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:45:48.372272 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:45:48.372272 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:45:48.372272 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:45:48.372272 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:45:48.373094 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:45:48.373094 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:45:48.373094 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:45:48.373094 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:45:48.373094 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 09:45:48.373094 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 09:45:48.373094 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 09:45:48.373094 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 09:45:48.373094 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 09:45:48.373094 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 09:45:48.373094 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 09:45:48.373094 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:45:48.373094 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 115/230] 10.222.0.64 () {56 vars in 1114 bytes} [Tue Aug 2 09:45:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 09:45:48.380639 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b921c019-b2e0-4c51-9717-345589111a59 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:48.490209 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:48.500656 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b921c019-b2e0-4c51-9717-345589111a59 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:48.540663 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2845e1eb-12e9-4a32-a052-7c5bf3035784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:48.541297 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2845e1eb-12e9-4a32-a052-7c5bf3035784 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:48.541583 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2845e1eb-12e9-4a32-a052-7c5bf3035784 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:48.541864 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2845e1eb-12e9-4a32-a052-7c5bf3035784 None admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:48.542668 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2845e1eb-12e9-4a32-a052-7c5bf3035784 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:45:48.542983 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2845e1eb-12e9-4a32-a052-7c5bf3035784 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:48.544406 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2845e1eb-12e9-4a32-a052-7c5bf3035784 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:48.553553 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 116/231] 10.222.0.64 () {56 vars in 1111 bytes} [Tue Aug 2 09:45:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:45:48.559005 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf5a5f1-c0bc-44f1-b251-b8780458fc46 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:48.668739 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf5a5f1-c0bc-44f1-b251-b8780458fc46 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:48.704900 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16b0680-3dae-493c-8df9-402ef7d6f8b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:48.705552 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16b0680-3dae-493c-8df9-402ef7d6f8b7 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:48.705839 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16b0680-3dae-493c-8df9-402ef7d6f8b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:48.706663 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16b0680-3dae-493c-8df9-402ef7d6f8b7 None admin] PATH_INFO: `/v3/users/3a33da6a369a4c3b9f5a663e28f6d306` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:48.707870 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16b0680-3dae-493c-8df9-402ef7d6f8b7 None admin] RBAC: Authorizing `identity:get_user(user_id=3a33da6a369a4c3b9f5a663e28f6d306)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:48.709448 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16b0680-3dae-493c-8df9-402ef7d6f8b7 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:48.711576 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 116/232] 10.222.0.64 () {56 vars in 1177 bytes} [Tue Aug 2 09:45:48 2022] GET /identity/v3/users/3a33da6a369a4c3b9f5a663e28f6d306 => generated 283 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:45:48.716937 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2845e1eb-12e9-4a32-a052-7c5bf3035784 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:48.826581 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2845e1eb-12e9-4a32-a052-7c5bf3035784 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:48.862816 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e09ea71-156e-47e1-84ad-b26234657387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:48.863532 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e09ea71-156e-47e1-84ad-b26234657387 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:48.863820 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e09ea71-156e-47e1-84ad-b26234657387 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:48.864097 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e09ea71-156e-47e1-84ad-b26234657387 None admin] PATH_INFO: `/v3/projects/bc7fa79dde7e42ae92a2a42495efdcc2` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:48.865401 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e09ea71-156e-47e1-84ad-b26234657387 None admin] RBAC: Authorizing `identity:get_project(project_id=bc7fa79dde7e42ae92a2a42495efdcc2)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:48.866927 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e09ea71-156e-47e1-84ad-b26234657387 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:48.869145 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 117/233] 10.222.0.64 () {56 vars in 1186 bytes} [Tue Aug 2 09:45:48 2022] GET /identity/v3/projects/bc7fa79dde7e42ae92a2a42495efdcc2 => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:45:48.877364 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16b0680-3dae-493c-8df9-402ef7d6f8b7 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:48.929691 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:49.006535 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16b0680-3dae-493c-8df9-402ef7d6f8b7 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:49.041860 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5463dfb-e139-4f5b-a5b9-4aa65ab01471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:49.042537 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5463dfb-e139-4f5b-a5b9-4aa65ab01471 None admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:49.042823 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5463dfb-e139-4f5b-a5b9-4aa65ab01471 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:49.043105 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5463dfb-e139-4f5b-a5b9-4aa65ab01471 None admin] PATH_INFO: `/v3/projects/bc7fa79dde7e42ae92a2a42495efdcc2/users/3a33da6a369a4c3b9f5a663e28f6d306/roles/06dfd237063c42a4bff1ad8ca29c07b0` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:49.045467 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5463dfb-e139-4f5b-a5b9-4aa65ab01471 None admin] RBAC: Authorizing `identity:create_grant(project_id=bc7fa79dde7e42ae92a2a42495efdcc2, user_id=3a33da6a369a4c3b9f5a663e28f6d306, role_id=06dfd237063c42a4bff1ad8ca29c07b0)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:49.046990 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5463dfb-e139-4f5b-a5b9-4aa65ab01471 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:49.089641 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 117/234] 10.222.0.64 () {58 vars in 1439 bytes} [Tue Aug 2 09:45:48 2022] PUT /identity/v3/projects/bc7fa79dde7e42ae92a2a42495efdcc2/users/3a33da6a369a4c3b9f5a663e28f6d306/roles/06dfd237063c42a4bff1ad8ca29c07b0 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:45:49.281974 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267454e6-59c4-4f40-8f62-081b105ddef6 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:49.282343 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267454e6-59c4-4f40-8f62-081b105ddef6 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:49.282539 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267454e6-59c4-4f40-8f62-081b105ddef6 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:49.283918 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 118/235] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:45:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:45:49.290264 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e940d32f-8163-416f-a3c9-089c8b4ccc1b None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:49.290569 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e940d32f-8163-416f-a3c9-089c8b4ccc1b None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:49.290900 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e940d32f-8163-416f-a3c9-089c8b4ccc1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:49.315695 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-e940d32f-8163-416f-a3c9-089c8b4ccc1b None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:45:49.316959 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-e940d32f-8163-416f-a3c9-089c8b4ccc1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:45:49.389666 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b3260e-069c-4c54-b8ad-7587a5045736 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:49.389861 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b3260e-069c-4c54-b8ad-7587a5045736 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:49.390152 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b3260e-069c-4c54-b8ad-7587a5045736 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:49.391875 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 119/236] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:45:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:45:49.393268 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 118/237] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:45:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 09:45:49.397106 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82e6dc4-fa75-40b3-a48d-c9670b5b530c None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:49.397399 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82e6dc4-fa75-40b3-a48d-c9670b5b530c None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:49.397680 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82e6dc4-fa75-40b3-a48d-c9670b5b530c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:49.418754 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-c82e6dc4-fa75-40b3-a48d-c9670b5b530c None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:45:49.420064 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-c82e6dc4-fa75-40b3-a48d-c9670b5b530c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:45:49.420313 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e940d32f-8163-416f-a3c9-089c8b4ccc1b None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:49.436363 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:49.459290 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:49.498632 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 120/238] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:45:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 09:45:49.522880 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82e6dc4-fa75-40b3-a48d-c9670b5b530c None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:49.547278 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e940d32f-8163-416f-a3c9-089c8b4ccc1b None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:49.590291 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781c0ef4-f0d1-40d3-af70-d079f3d10240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:49.590953 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781c0ef4-f0d1-40d3-af70-d079f3d10240 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:49.591365 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781c0ef4-f0d1-40d3-af70-d079f3d10240 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:49.591710 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781c0ef4-f0d1-40d3-af70-d079f3d10240 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:49.592972 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781c0ef4-f0d1-40d3-af70-d079f3d10240 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:49.594499 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781c0ef4-f0d1-40d3-af70-d079f3d10240 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:49.596213 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 119/239] 10.222.0.64 () {56 vars in 1108 bytes} [Tue Aug 2 09:45:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:45:49.602690 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781c0ef4-f0d1-40d3-af70-d079f3d10240 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:49.637451 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82e6dc4-fa75-40b3-a48d-c9670b5b530c None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:49.673720 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525f89ec-95d4-45c6-942e-4447925259d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:49.674452 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525f89ec-95d4-45c6-942e-4447925259d1 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:49.674845 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525f89ec-95d4-45c6-942e-4447925259d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:49.675196 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525f89ec-95d4-45c6-942e-4447925259d1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:49.685802 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525f89ec-95d4-45c6-942e-4447925259d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 09:45:49.691085 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-525f89ec-95d4-45c6-942e-4447925259d1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 02 09:45:49.691085 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:45:49.691085 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:45:49.691085 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:45:49.691085 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:45:49.691085 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:45:49.691085 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:45:49.691085 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:45:49.691085 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:45:49.691085 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:45:49.691085 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:45:49.691085 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:45:49.691085 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 09:45:49.691085 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 09:45:49.691085 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 09:45:49.691085 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 09:45:49.692005 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 09:45:49.692005 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 09:45:49.692005 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 09:45:49.692005 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 09:45:49.692005 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:45:49.692005 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:45:49.692005 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:45:49.692005 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:45:49.692005 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:45:49.692005 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:45:49.692005 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:45:49.692005 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:45:49.692005 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:45:49.692005 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:45:49.692005 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:45:49.692005 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:45:49.692842 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:45:49.692842 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:45:49.692842 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:45:49.692842 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:45:49.692842 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 09:45:49.692842 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 09:45:49.692842 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 09:45:49.692842 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 09:45:49.692842 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 09:45:49.692842 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 09:45:49.692842 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 02 09:45:49.692842 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:45:49.692842 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 121/240] 10.222.0.64 () {56 vars in 1099 bytes} [Tue Aug 2 09:45:49 2022] GET /identity/v3/roles/reader => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 09:45:49.699621 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525f89ec-95d4-45c6-942e-4447925259d1 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:49.716574 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:49.732716 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781c0ef4-f0d1-40d3-af70-d079f3d10240 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:49.734472 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:49.771103 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8524feca-cb1f-45d5-8f7d-4cf729d3f51e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:49.771880 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8524feca-cb1f-45d5-8f7d-4cf729d3f51e None admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:49.772259 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8524feca-cb1f-45d5-8f7d-4cf729d3f51e None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:49.772650 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8524feca-cb1f-45d5-8f7d-4cf729d3f51e None admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:49.773483 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8524feca-cb1f-45d5-8f7d-4cf729d3f51e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:49.775084 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8524feca-cb1f-45d5-8f7d-4cf729d3f51e None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:49.805903 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 120/241] 10.222.0.64 () {60 vars in 1132 bytes} [Tue Aug 2 09:45:49 2022] POST /identity/v3/users => generated 293 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:45:49.879388 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525f89ec-95d4-45c6-942e-4447925259d1 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:49.931128 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8ba9c6-8d9d-4fc5-a5bc-5c40dd58fd5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:49.931680 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8ba9c6-8d9d-4fc5-a5bc-5c40dd58fd5a None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:49.931965 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8ba9c6-8d9d-4fc5-a5bc-5c40dd58fd5a None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:49.932260 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8ba9c6-8d9d-4fc5-a5bc-5c40dd58fd5a None admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:49.933008 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8ba9c6-8d9d-4fc5-a5bc-5c40dd58fd5a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:45:49.933314 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8ba9c6-8d9d-4fc5-a5bc-5c40dd58fd5a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:49.934736 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8ba9c6-8d9d-4fc5-a5bc-5c40dd58fd5a None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:49.948790 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 122/242] 10.222.0.64 () {56 vars in 1101 bytes} [Tue Aug 2 09:45:49 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:45:49.954618 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8524feca-cb1f-45d5-8f7d-4cf729d3f51e None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:50.088574 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8524feca-cb1f-45d5-8f7d-4cf729d3f51e None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:50.116280 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624d0100-0ea9-4b1e-9d6a-6a59d2f14fd1 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:50.116606 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624d0100-0ea9-4b1e-9d6a-6a59d2f14fd1 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:50.116877 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624d0100-0ea9-4b1e-9d6a-6a59d2f14fd1 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:50.118402 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 123/243] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:45:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:45:50.124214 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137da548-1fb4-4b3d-96ef-767ea462d471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:50.124901 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137da548-1fb4-4b3d-96ef-767ea462d471 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:50.125188 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137da548-1fb4-4b3d-96ef-767ea462d471 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:50.125308 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb58980-e700-4d22-984f-e09f8291de21 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:50.125474 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137da548-1fb4-4b3d-96ef-767ea462d471 None admin] PATH_INFO: `/v3/users/ad730f60ea8e41dca4715c61f0050afa` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:50.125564 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb58980-e700-4d22-984f-e09f8291de21 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:50.125748 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb58980-e700-4d22-984f-e09f8291de21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:50.142794 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137da548-1fb4-4b3d-96ef-767ea462d471 None admin] RBAC: Authorizing `identity:get_user(user_id=ad730f60ea8e41dca4715c61f0050afa)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:50.145024 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137da548-1fb4-4b3d-96ef-767ea462d471 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:50.146835 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 121/244] 10.222.0.64 () {56 vars in 1177 bytes} [Tue Aug 2 09:45:49 2022] GET /identity/v3/users/ad730f60ea8e41dca4715c61f0050afa => generated 297 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:45:50.152943 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137da548-1fb4-4b3d-96ef-767ea462d471 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:50.158807 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-acb58980-e700-4d22-984f-e09f8291de21 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:45:50.160048 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-acb58980-e700-4d22-984f-e09f8291de21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:45:50.169620 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:50.270581 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 124/245] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:45:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 09:45:50.279500 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137da548-1fb4-4b3d-96ef-767ea462d471 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:50.296049 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb58980-e700-4d22-984f-e09f8291de21 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:50.298250 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:50.330763 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74818497-83d5-4061-8679-728ba0e5491b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:50.331915 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74818497-83d5-4061-8679-728ba0e5491b None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:50.332441 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74818497-83d5-4061-8679-728ba0e5491b None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:50.332920 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74818497-83d5-4061-8679-728ba0e5491b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:50.432013 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74818497-83d5-4061-8679-728ba0e5491b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:50.432013 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74818497-83d5-4061-8679-728ba0e5491b None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:50.436346 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb58980-e700-4d22-984f-e09f8291de21 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:50.444191 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:50.480832 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3928a54e-b27a-4012-aaeb-13dcc02f789d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:50.481856 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3928a54e-b27a-4012-aaeb-13dcc02f789d None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:50.482433 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3928a54e-b27a-4012-aaeb-13dcc02f789d None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:50.482919 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3928a54e-b27a-4012-aaeb-13dcc02f789d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:50.495843 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3928a54e-b27a-4012-aaeb-13dcc02f789d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 09:45:50.501681 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-3928a54e-b27a-4012-aaeb-13dcc02f789d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 02 09:45:50.501681 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:45:50.501681 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:45:50.501681 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:45:50.501681 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:45:50.501681 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:45:50.501681 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:45:50.501681 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:45:50.501681 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:45:50.501681 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:45:50.501681 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:45:50.501681 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:45:50.501681 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 09:45:50.501681 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 09:45:50.501681 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 09:45:50.501681 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 09:45:50.503117 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 09:45:50.503117 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 09:45:50.503117 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 09:45:50.503117 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 09:45:50.503117 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:45:50.503117 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:45:50.503117 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:45:50.503117 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:45:50.503117 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:45:50.503117 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:45:50.503117 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:45:50.503117 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:45:50.503117 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:45:50.503117 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:45:50.503117 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:45:50.503117 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:45:50.503935 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:45:50.503935 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:45:50.503935 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:45:50.503935 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:45:50.503935 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 09:45:50.503935 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 09:45:50.503935 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 09:45:50.503935 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 09:45:50.503935 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 09:45:50.503935 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 09:45:50.503935 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 02 09:45:50.503935 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:45:50.506543 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 125/246] 10.222.0.64 () {56 vars in 1099 bytes} [Tue Aug 2 09:45:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 09:45:50.512358 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3928a54e-b27a-4012-aaeb-13dcc02f789d None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:50.557640 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 122/247] 10.222.0.64 () {58 vars in 1282 bytes} [Tue Aug 2 09:45:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 09:45:50.563620 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74818497-83d5-4061-8679-728ba0e5491b None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:50.589798 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:50.640813 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3928a54e-b27a-4012-aaeb-13dcc02f789d None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:50.698926 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a43690-c6de-46f0-9368-21faa9d6879d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:50.699952 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a43690-c6de-46f0-9368-21faa9d6879d None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:50.701776 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a43690-c6de-46f0-9368-21faa9d6879d None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:50.702259 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a43690-c6de-46f0-9368-21faa9d6879d None admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:50.703186 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a43690-c6de-46f0-9368-21faa9d6879d None admin] RBAC: Adding query filter params (name=member) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:45:50.704436 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a43690-c6de-46f0-9368-21faa9d6879d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:50.707200 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a43690-c6de-46f0-9368-21faa9d6879d None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:50.710938 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74818497-83d5-4061-8679-728ba0e5491b None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:50.721139 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 126/248] 10.222.0.64 () {56 vars in 1101 bytes} [Tue Aug 2 09:45:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:45:50.725087 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:50.728457 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a43690-c6de-46f0-9368-21faa9d6879d None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:50.781389 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70cd3f1-40ec-41b3-94c7-85fa6c3cc090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:50.782416 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70cd3f1-40ec-41b3-94c7-85fa6c3cc090 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:50.782901 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70cd3f1-40ec-41b3-94c7-85fa6c3cc090 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:50.783376 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70cd3f1-40ec-41b3-94c7-85fa6c3cc090 None admin] PATH_INFO: `/v3/projects/bc7fa79dde7e42ae92a2a42495efdcc2` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:50.786449 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70cd3f1-40ec-41b3-94c7-85fa6c3cc090 None admin] RBAC: Authorizing `identity:get_project(project_id=bc7fa79dde7e42ae92a2a42495efdcc2)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:50.788125 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70cd3f1-40ec-41b3-94c7-85fa6c3cc090 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:50.793166 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 123/249] 10.222.0.64 () {56 vars in 1186 bytes} [Tue Aug 2 09:45:50 2022] GET /identity/v3/projects/bc7fa79dde7e42ae92a2a42495efdcc2 => generated 302 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:45:50.798502 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70cd3f1-40ec-41b3-94c7-85fa6c3cc090 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:50.919076 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a43690-c6de-46f0-9368-21faa9d6879d None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:50.974313 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:50.986755 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70cd3f1-40ec-41b3-94c7-85fa6c3cc090 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:51.033568 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440d24d9-6a85-4225-85ca-5716e715cacd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:51.034282 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440d24d9-6a85-4225-85ca-5716e715cacd None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:51.034573 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440d24d9-6a85-4225-85ca-5716e715cacd None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:51.034858 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440d24d9-6a85-4225-85ca-5716e715cacd None admin] PATH_INFO: `/v3/users/31178481748442ada86cab1c8bc75b18` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:51.036075 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440d24d9-6a85-4225-85ca-5716e715cacd None admin] RBAC: Authorizing `identity:get_user(user_id=31178481748442ada86cab1c8bc75b18)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:51.037574 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440d24d9-6a85-4225-85ca-5716e715cacd None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:51.039387 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 127/250] 10.222.0.64 () {56 vars in 1177 bytes} [Tue Aug 2 09:45:50 2022] GET /identity/v3/users/31178481748442ada86cab1c8bc75b18 => generated 297 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:45:51.044694 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440d24d9-6a85-4225-85ca-5716e715cacd None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:51.047079 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e47908-a225-455b-a9cd-61689c95e8ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:51.047710 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e47908-a225-455b-a9cd-61689c95e8ae None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:51.047990 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e47908-a225-455b-a9cd-61689c95e8ae None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:51.048279 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e47908-a225-455b-a9cd-61689c95e8ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:51.049161 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e47908-a225-455b-a9cd-61689c95e8ae None admin] RBAC: Adding query filter params (role.id=7fce0045dfa049bdb7ae0d2424de9f32, scope.project.id=bc7fa79dde7e42ae92a2a42495efdcc2, user.id=ad730f60ea8e41dca4715c61f0050afa) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:45:51.049476 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e47908-a225-455b-a9cd-61689c95e8ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:51.051698 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e47908-a225-455b-a9cd-61689c95e8ae None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:51.063449 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 124/251] 10.222.0.64 () {56 vars in 1374 bytes} [Tue Aug 2 09:45:50 2022] GET /identity/v3/role_assignments?user.id=ad730f60ea8e41dca4715c61f0050afa&scope.project.id=bc7fa79dde7e42ae92a2a42495efdcc2&role.id=7fce0045dfa049bdb7ae0d2424de9f32 => generated 259 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:45:51.133249 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a59afdb-da1c-4123-978d-e034337748e2 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:51.133566 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a59afdb-da1c-4123-978d-e034337748e2 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:51.133812 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a59afdb-da1c-4123-978d-e034337748e2 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:51.135216 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 125/252] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:45:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 09:45:51.146640 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc18c30d-7a11-49da-bf96-2fb2d2372447 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:51.146902 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc18c30d-7a11-49da-bf96-2fb2d2372447 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:51.147120 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc18c30d-7a11-49da-bf96-2fb2d2372447 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:51.182372 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-fc18c30d-7a11-49da-bf96-2fb2d2372447 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:45:51.183596 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-fc18c30d-7a11-49da-bf96-2fb2d2372447 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:45:51.223776 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440d24d9-6a85-4225-85ca-5716e715cacd None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:51.241925 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:51.274975 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 126/253] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:45:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 09:45:51.300590 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc18c30d-7a11-49da-bf96-2fb2d2372447 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:51.303784 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc16060-03a4-4d71-b0c2-988a88e3f706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:51.304483 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc16060-03a4-4d71-b0c2-988a88e3f706 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:51.304774 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc16060-03a4-4d71-b0c2-988a88e3f706 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:51.305052 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc16060-03a4-4d71-b0c2-988a88e3f706 None admin] PATH_INFO: `/v3/projects/bc7fa79dde7e42ae92a2a42495efdcc2` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:51.309020 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc16060-03a4-4d71-b0c2-988a88e3f706 None admin] RBAC: Authorizing `identity:get_project(project_id=bc7fa79dde7e42ae92a2a42495efdcc2)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:51.310550 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc16060-03a4-4d71-b0c2-988a88e3f706 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:51.312409 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 128/254] 10.222.0.64 () {56 vars in 1186 bytes} [Tue Aug 2 09:45:51 2022] GET /identity/v3/projects/bc7fa79dde7e42ae92a2a42495efdcc2 => generated 302 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:45:51.321684 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc16060-03a4-4d71-b0c2-988a88e3f706 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:51.489094 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc18c30d-7a11-49da-bf96-2fb2d2372447 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:51.491287 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:51.526590 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715d6244-cd50-4563-8628-53ee2ee0bc10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:51.527565 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715d6244-cd50-4563-8628-53ee2ee0bc10 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:51.528555 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715d6244-cd50-4563-8628-53ee2ee0bc10 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:51.528982 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715d6244-cd50-4563-8628-53ee2ee0bc10 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:51.541875 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715d6244-cd50-4563-8628-53ee2ee0bc10 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 09:45:51.547609 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-715d6244-cd50-4563-8628-53ee2ee0bc10 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 09:45:51.547609 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:45:51.547609 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:45:51.547609 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:45:51.547609 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:45:51.547609 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:45:51.547609 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:45:51.547609 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:45:51.547609 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:45:51.547609 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:45:51.547609 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:45:51.547609 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:45:51.547609 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 09:45:51.547609 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 09:45:51.547609 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 09:45:51.547609 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 09:45:51.548557 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 09:45:51.548557 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 09:45:51.548557 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 09:45:51.548557 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 09:45:51.548557 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:45:51.548557 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:45:51.548557 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:45:51.548557 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:45:51.548557 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:45:51.548557 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:45:51.548557 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:45:51.548557 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:45:51.548557 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:45:51.548557 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:45:51.548557 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:45:51.548557 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:45:51.549384 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:45:51.549384 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:45:51.549384 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:45:51.549384 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:45:51.549384 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 09:45:51.549384 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 09:45:51.549384 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 09:45:51.549384 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 09:45:51.549384 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 09:45:51.549384 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 09:45:51.549384 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 09:45:51.549384 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:45:51.556891 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 127/255] 10.222.0.64 () {56 vars in 1096 bytes} [Tue Aug 2 09:45:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:45:51.559011 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715d6244-cd50-4563-8628-53ee2ee0bc10 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:51.603832 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:51.658202 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc16060-03a4-4d71-b0c2-988a88e3f706 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:51.679465 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715d6244-cd50-4563-8628-53ee2ee0bc10 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:51.736942 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1639d45-362a-4425-ba7e-3ebc2fe6a746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:51.738187 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6518462-925b-4792-a235-b7d54161a63b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:51.738975 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6518462-925b-4792-a235-b7d54161a63b None admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:51.739205 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1639d45-362a-4425-ba7e-3ebc2fe6a746 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:51.739431 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6518462-925b-4792-a235-b7d54161a63b None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:51.739800 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6518462-925b-4792-a235-b7d54161a63b None admin] PATH_INFO: `/v3/projects/bc7fa79dde7e42ae92a2a42495efdcc2/users/31178481748442ada86cab1c8bc75b18/roles/66c1074780954f8e87ed11a5c4e8c36a` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:51.739938 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1639d45-362a-4425-ba7e-3ebc2fe6a746 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:51.740368 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1639d45-362a-4425-ba7e-3ebc2fe6a746 None admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:51.741230 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1639d45-362a-4425-ba7e-3ebc2fe6a746 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:45:51.741680 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1639d45-362a-4425-ba7e-3ebc2fe6a746 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:51.743188 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1639d45-362a-4425-ba7e-3ebc2fe6a746 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:51.745943 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6518462-925b-4792-a235-b7d54161a63b None admin] RBAC: Authorizing `identity:create_grant(project_id=bc7fa79dde7e42ae92a2a42495efdcc2, user_id=31178481748442ada86cab1c8bc75b18, role_id=66c1074780954f8e87ed11a5c4e8c36a)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:51.747271 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6518462-925b-4792-a235-b7d54161a63b None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:51.756649 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 128/256] 10.222.0.64 () {56 vars in 1099 bytes} [Tue Aug 2 09:45:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:45:51.762604 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1639d45-362a-4425-ba7e-3ebc2fe6a746 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:51.782749 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 129/257] 10.222.0.64 () {58 vars in 1439 bytes} [Tue Aug 2 09:45:51 2022] PUT /identity/v3/projects/bc7fa79dde7e42ae92a2a42495efdcc2/users/31178481748442ada86cab1c8bc75b18/roles/66c1074780954f8e87ed11a5c4e8c36a => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:45:51.823792 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e76a9d5-1539-4af3-829c-4a143bf303c7 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:51.824142 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e76a9d5-1539-4af3-829c-4a143bf303c7 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:51.824386 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e76a9d5-1539-4af3-829c-4a143bf303c7 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:51.825781 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 130/258] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:45:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:45:51.835485 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4903bb1-3736-46ff-8f7c-960ffa0bb750 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:51.835776 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4903bb1-3736-46ff-8f7c-960ffa0bb750 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:51.836042 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4903bb1-3736-46ff-8f7c-960ffa0bb750 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:51.872940 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-e4903bb1-3736-46ff-8f7c-960ffa0bb750 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:45:51.874176 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-e4903bb1-3736-46ff-8f7c-960ffa0bb750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:45:51.952374 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:51.979916 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1639d45-362a-4425-ba7e-3ebc2fe6a746 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:52.054575 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 131/259] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:45:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 09:45:52.062369 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e47bce8-6c69-4af1-8ed3-9eaa86ab7fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:52.063042 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e47bce8-6c69-4af1-8ed3-9eaa86ab7fac None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:52.063324 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e47bce8-6c69-4af1-8ed3-9eaa86ab7fac None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:52.063600 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e47bce8-6c69-4af1-8ed3-9eaa86ab7fac None admin] PATH_INFO: `/v3/users/3a33da6a369a4c3b9f5a663e28f6d306` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:52.069038 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e47bce8-6c69-4af1-8ed3-9eaa86ab7fac None admin] RBAC: Authorizing `identity:get_user(user_id=3a33da6a369a4c3b9f5a663e28f6d306)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:52.070510 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e47bce8-6c69-4af1-8ed3-9eaa86ab7fac None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:52.072470 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 129/260] 10.222.0.64 () {56 vars in 1177 bytes} [Tue Aug 2 09:45:51 2022] GET /identity/v3/users/3a33da6a369a4c3b9f5a663e28f6d306 => generated 283 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:45:52.078243 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4903bb1-3736-46ff-8f7c-960ffa0bb750 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:52.099695 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e47bce8-6c69-4af1-8ed3-9eaa86ab7fac None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:52.157148 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:52.211457 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4903bb1-3736-46ff-8f7c-960ffa0bb750 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:52.266942 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bdee12-ca7c-4971-90a7-c6741f93d4b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:52.267571 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bdee12-ca7c-4971-90a7-c6741f93d4b0 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:52.267859 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bdee12-ca7c-4971-90a7-c6741f93d4b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:52.268358 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bdee12-ca7c-4971-90a7-c6741f93d4b0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:52.279046 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e47bce8-6c69-4af1-8ed3-9eaa86ab7fac None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:52.337956 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba496472-a753-4ae4-b24f-da4d499c55c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:52.338639 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba496472-a753-4ae4-b24f-da4d499c55c3 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:52.338923 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba496472-a753-4ae4-b24f-da4d499c55c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:52.339201 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba496472-a753-4ae4-b24f-da4d499c55c3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:52.357133 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba496472-a753-4ae4-b24f-da4d499c55c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 09:45:52.369051 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-ba496472-a753-4ae4-b24f-da4d499c55c3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 09:45:52.369051 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:45:52.369051 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:45:52.369051 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:45:52.369051 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:45:52.369051 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:45:52.369051 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:45:52.369051 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:45:52.369051 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:45:52.369051 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:45:52.369051 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:45:52.369051 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:45:52.369051 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 09:45:52.369051 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 09:45:52.369051 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 09:45:52.369051 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 09:45:52.369998 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 09:45:52.369998 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 09:45:52.369998 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 09:45:52.369998 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 09:45:52.369998 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:45:52.369998 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:45:52.369998 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:45:52.369998 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:45:52.369998 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:45:52.369998 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:45:52.369998 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:45:52.369998 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:45:52.369998 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:45:52.369998 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:45:52.369998 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:45:52.369998 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:45:52.370813 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:45:52.370813 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:45:52.370813 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:45:52.370813 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:45:52.370813 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 09:45:52.370813 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 09:45:52.370813 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 09:45:52.370813 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 09:45:52.370813 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 09:45:52.370813 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 09:45:52.370813 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 09:45:52.370813 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:45:52.370813 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 130/261] 10.222.0.64 () {56 vars in 1114 bytes} [Tue Aug 2 09:45:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:45:52.371478 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bdee12-ca7c-4971-90a7-c6741f93d4b0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:52.372773 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bdee12-ca7c-4971-90a7-c6741f93d4b0 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:52.375901 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba496472-a753-4ae4-b24f-da4d499c55c3 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:52.414032 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:52.468008 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:52.520928 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba496472-a753-4ae4-b24f-da4d499c55c3 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:52.558508 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff5f510-8a16-4eaa-8825-bc6becc6ed4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:52.559146 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff5f510-8a16-4eaa-8825-bc6becc6ed4e None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:52.559433 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff5f510-8a16-4eaa-8825-bc6becc6ed4e None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:52.559712 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff5f510-8a16-4eaa-8825-bc6becc6ed4e None admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:52.560481 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff5f510-8a16-4eaa-8825-bc6becc6ed4e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:45:52.561822 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff5f510-8a16-4eaa-8825-bc6becc6ed4e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:52.563254 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff5f510-8a16-4eaa-8825-bc6becc6ed4e None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:52.584978 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 131/262] 10.222.0.64 () {56 vars in 1111 bytes} [Tue Aug 2 09:45:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:45:52.590844 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff5f510-8a16-4eaa-8825-bc6becc6ed4e None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:52.593984 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 132/263] 10.222.0.64 () {58 vars in 1282 bytes} [Tue Aug 2 09:45:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 516 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 09:45:52.600660 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bdee12-ca7c-4971-90a7-c6741f93d4b0 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:52.761787 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:52.783067 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff5f510-8a16-4eaa-8825-bc6becc6ed4e None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:52.822190 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccca8ef0-7240-4e4a-9f05-4c8443f3d58b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:52.822848 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccca8ef0-7240-4e4a-9f05-4c8443f3d58b None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:52.823139 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccca8ef0-7240-4e4a-9f05-4c8443f3d58b None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:52.823421 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccca8ef0-7240-4e4a-9f05-4c8443f3d58b None admin] PATH_INFO: `/v3/users/3a33da6a369a4c3b9f5a663e28f6d306` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:52.824670 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccca8ef0-7240-4e4a-9f05-4c8443f3d58b None admin] RBAC: Authorizing `identity:get_user(user_id=3a33da6a369a4c3b9f5a663e28f6d306)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:52.826155 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccca8ef0-7240-4e4a-9f05-4c8443f3d58b None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:52.828202 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 132/264] 10.222.0.64 () {56 vars in 1177 bytes} [Tue Aug 2 09:45:52 2022] GET /identity/v3/users/3a33da6a369a4c3b9f5a663e28f6d306 => generated 283 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:45:52.837360 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccca8ef0-7240-4e4a-9f05-4c8443f3d58b None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:52.907178 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bdee12-ca7c-4971-90a7-c6741f93d4b0 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:52.994346 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccca8ef0-7240-4e4a-9f05-4c8443f3d58b None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:53.016982 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5268ab17-64b8-4fa0-9fa9-f79b3d05972d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:53.017654 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5268ab17-64b8-4fa0-9fa9-f79b3d05972d None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:53.017940 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5268ab17-64b8-4fa0-9fa9-f79b3d05972d None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:53.018246 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5268ab17-64b8-4fa0-9fa9-f79b3d05972d None admin] PATH_INFO: `/v3/projects/bc7fa79dde7e42ae92a2a42495efdcc2` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:53.029148 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5268ab17-64b8-4fa0-9fa9-f79b3d05972d None admin] RBAC: Authorizing `identity:get_project(project_id=bc7fa79dde7e42ae92a2a42495efdcc2)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:53.030649 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5268ab17-64b8-4fa0-9fa9-f79b3d05972d None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:53.032807 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 133/265] 10.222.0.64 () {56 vars in 1186 bytes} [Tue Aug 2 09:45:52 2022] GET /identity/v3/projects/bc7fa79dde7e42ae92a2a42495efdcc2 => generated 302 bytes in 432 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:45:53.041268 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5268ab17-64b8-4fa0-9fa9-f79b3d05972d None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:53.044997 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47178326-e2c3-47ea-a21b-0bb6147f0cfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:53.045655 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47178326-e2c3-47ea-a21b-0bb6147f0cfe None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:53.045947 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47178326-e2c3-47ea-a21b-0bb6147f0cfe None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:53.046226 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47178326-e2c3-47ea-a21b-0bb6147f0cfe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:53.079615 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:53.095162 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:53.222332 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5268ab17-64b8-4fa0-9fa9-f79b3d05972d None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:53.257151 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47178326-e2c3-47ea-a21b-0bb6147f0cfe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:53.258659 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47178326-e2c3-47ea-a21b-0bb6147f0cfe None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:53.281693 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c53a7c2-b4c2-4fb2-af68-8296d3cf03c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:53.282342 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c53a7c2-b4c2-4fb2-af68-8296d3cf03c4 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:53.282625 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c53a7c2-b4c2-4fb2-af68-8296d3cf03c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:53.282899 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c53a7c2-b4c2-4fb2-af68-8296d3cf03c4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:53.283712 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c53a7c2-b4c2-4fb2-af68-8296d3cf03c4 None admin] RBAC: Adding query filter params (role.id=1000cf318fa14915b8091599eaead5de, scope.project.id=bc7fa79dde7e42ae92a2a42495efdcc2, user.id=3a33da6a369a4c3b9f5a663e28f6d306) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:45:53.284057 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c53a7c2-b4c2-4fb2-af68-8296d3cf03c4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:53.285567 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c53a7c2-b4c2-4fb2-af68-8296d3cf03c4 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:53.300767 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 134/266] 10.222.0.64 () {56 vars in 1374 bytes} [Tue Aug 2 09:45:53 2022] GET /identity/v3/role_assignments?user.id=3a33da6a369a4c3b9f5a663e28f6d306&scope.project.id=bc7fa79dde7e42ae92a2a42495efdcc2&role.id=1000cf318fa14915b8091599eaead5de => generated 608 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:45:53.400773 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4ec72c-6339-4c6f-b689-81aeb3882b44 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:53.401094 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4ec72c-6339-4c6f-b689-81aeb3882b44 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:53.401334 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4ec72c-6339-4c6f-b689-81aeb3882b44 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:53.402968 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 135/267] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:45:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:45:53.413753 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1177d782-119a-4155-ae65-6eb8ff3988f1 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:53.414054 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1177d782-119a-4155-ae65-6eb8ff3988f1 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:53.414314 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1177d782-119a-4155-ae65-6eb8ff3988f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:53.442807 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 133/268] 10.222.0.64 () {58 vars in 1282 bytes} [Tue Aug 2 09:45:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 606 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 09:45:53.453952 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47178326-e2c3-47ea-a21b-0bb6147f0cfe None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:53.465448 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-1177d782-119a-4155-ae65-6eb8ff3988f1 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:45:53.466653 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-1177d782-119a-4155-ae65-6eb8ff3988f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:45:53.560688 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:53.644588 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47178326-e2c3-47ea-a21b-0bb6147f0cfe None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:53.658844 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 136/269] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:45:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 09:45:53.661110 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6ea3a7-1120-4039-b65a-e742321c8e54 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:53.661403 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6ea3a7-1120-4039-b65a-e742321c8e54 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:53.661681 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6ea3a7-1120-4039-b65a-e742321c8e54 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:53.663546 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 137/270] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:45:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:45:53.669213 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347d8ba2-ba22-43c3-8409-9a716cc220f0 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:53.669498 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347d8ba2-ba22-43c3-8409-9a716cc220f0 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:53.670408 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347d8ba2-ba22-43c3-8409-9a716cc220f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:53.695387 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-347d8ba2-ba22-43c3-8409-9a716cc220f0 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:45:53.697689 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-347d8ba2-ba22-43c3-8409-9a716cc220f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:45:53.710286 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:53.712473 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbc052b-22d7-42cb-b946-04128c75ca39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:53.713132 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbc052b-22d7-42cb-b946-04128c75ca39 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:53.713426 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbc052b-22d7-42cb-b946-04128c75ca39 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:53.713745 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbc052b-22d7-42cb-b946-04128c75ca39 None admin] PATH_INFO: `/v3/projects/bc7fa79dde7e42ae92a2a42495efdcc2` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:53.714952 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbc052b-22d7-42cb-b946-04128c75ca39 None admin] RBAC: Authorizing `identity:get_project(project_id=bc7fa79dde7e42ae92a2a42495efdcc2)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:53.716452 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbc052b-22d7-42cb-b946-04128c75ca39 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:53.718499 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 134/271] 10.222.0.64 () {56 vars in 1186 bytes} [Tue Aug 2 09:45:53 2022] GET /identity/v3/projects/bc7fa79dde7e42ae92a2a42495efdcc2 => generated 302 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:45:53.719302 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbc052b-22d7-42cb-b946-04128c75ca39 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:53.810637 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 138/272] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:45:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 09:45:53.811605 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347d8ba2-ba22-43c3-8409-9a716cc220f0 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:53.896184 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbc052b-22d7-42cb-b946-04128c75ca39 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:53.935795 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:53.941834 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f07de01-a07c-4189-b4db-6abd0db4beea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:53.942492 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f07de01-a07c-4189-b4db-6abd0db4beea None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:53.942774 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f07de01-a07c-4189-b4db-6abd0db4beea None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:53.943048 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f07de01-a07c-4189-b4db-6abd0db4beea None admin] PATH_INFO: `/v3/domains/default` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:53.944241 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f07de01-a07c-4189-b4db-6abd0db4beea None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:53.945726 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f07de01-a07c-4189-b4db-6abd0db4beea None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:53.952318 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 135/273] 10.222.0.64 () {56 vars in 1108 bytes} [Tue Aug 2 09:45:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:45:53.952318 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f07de01-a07c-4189-b4db-6abd0db4beea None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:54.001252 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347d8ba2-ba22-43c3-8409-9a716cc220f0 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:54.059921 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b5ef4f-bfcc-4e2a-bb8b-30b67d1425f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:54.060582 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b5ef4f-bfcc-4e2a-bb8b-30b67d1425f3 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:54.060866 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b5ef4f-bfcc-4e2a-bb8b-30b67d1425f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:54.061144 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b5ef4f-bfcc-4e2a-bb8b-30b67d1425f3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:54.062005 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b5ef4f-bfcc-4e2a-bb8b-30b67d1425f3 None admin] RBAC: Adding query filter params (role.id=06dfd237063c42a4bff1ad8ca29c07b0, scope.project.id=bc7fa79dde7e42ae92a2a42495efdcc2, user.id=3a33da6a369a4c3b9f5a663e28f6d306) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:45:54.062253 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b5ef4f-bfcc-4e2a-bb8b-30b67d1425f3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:54.063710 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b5ef4f-bfcc-4e2a-bb8b-30b67d1425f3 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:54.066467 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f07de01-a07c-4189-b4db-6abd0db4beea None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:54.070983 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 139/274] 10.222.0.64 () {56 vars in 1374 bytes} [Tue Aug 2 09:45:53 2022] GET /identity/v3/role_assignments?user.id=3a33da6a369a4c3b9f5a663e28f6d306&scope.project.id=bc7fa79dde7e42ae92a2a42495efdcc2&role.id=06dfd237063c42a4bff1ad8ca29c07b0 => generated 608 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:45:54.071804 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b5ef4f-bfcc-4e2a-bb8b-30b67d1425f3 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:54.081115 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:54.120815 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617dd383-72dc-41e2-ae21-4ab5e61541c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:54.121609 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617dd383-72dc-41e2-ae21-4ab5e61541c1 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:54.121950 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617dd383-72dc-41e2-ae21-4ab5e61541c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:54.122230 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617dd383-72dc-41e2-ae21-4ab5e61541c1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:54.136361 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617dd383-72dc-41e2-ae21-4ab5e61541c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 09:45:54.141767 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-617dd383-72dc-41e2-ae21-4ab5e61541c1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 02 09:45:54.141767 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:45:54.141767 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:45:54.141767 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:45:54.141767 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:45:54.141767 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:45:54.141767 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:45:54.141767 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:45:54.141767 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:45:54.141767 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:45:54.141767 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:45:54.141767 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:45:54.141767 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 09:45:54.141767 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 09:45:54.141767 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 09:45:54.141767 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 09:45:54.142733 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 09:45:54.142733 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 09:45:54.142733 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 09:45:54.142733 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 09:45:54.142733 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:45:54.142733 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:45:54.142733 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:45:54.142733 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:45:54.142733 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:45:54.142733 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:45:54.142733 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:45:54.142733 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:45:54.142733 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:45:54.142733 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:45:54.142733 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:45:54.142733 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:45:54.143612 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:45:54.143612 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:45:54.143612 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:45:54.143612 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:45:54.143612 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 09:45:54.143612 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 09:45:54.143612 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 09:45:54.143612 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 09:45:54.143612 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 09:45:54.143612 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 09:45:54.143612 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 02 09:45:54.143612 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:45:54.148168 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 136/275] 10.222.0.64 () {56 vars in 1099 bytes} [Tue Aug 2 09:45:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:45:54.149738 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86c11da-a70f-4bea-b3f4-47540c5cb208 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:54.150023 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86c11da-a70f-4bea-b3f4-47540c5cb208 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:54.150309 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86c11da-a70f-4bea-b3f4-47540c5cb208 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:54.152066 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 137/276] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:45:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:45:54.152865 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86c11da-a70f-4bea-b3f4-47540c5cb208 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:54.226957 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b5ef4f-bfcc-4e2a-bb8b-30b67d1425f3 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:54.233895 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:54.288426 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6fc761-a57b-41a5-94bb-e94c350a68ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:54.288633 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6fc761-a57b-41a5-94bb-e94c350a68ba None admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:54.288916 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6fc761-a57b-41a5-94bb-e94c350a68ba None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:54.289185 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6fc761-a57b-41a5-94bb-e94c350a68ba None admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:54.289884 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6fc761-a57b-41a5-94bb-e94c350a68ba None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:54.291348 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6fc761-a57b-41a5-94bb-e94c350a68ba None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:54.325138 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 140/277] 10.222.0.64 () {60 vars in 1132 bytes} [Tue Aug 2 09:45:54 2022] POST /identity/v3/users => generated 293 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:45:54.326430 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be783b6c-86b4-4e09-abdb-91606e0dd8d1 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:54.326722 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be783b6c-86b4-4e09-abdb-91606e0dd8d1 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:54.327060 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be783b6c-86b4-4e09-abdb-91606e0dd8d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:54.329133 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86c11da-a70f-4bea-b3f4-47540c5cb208 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:54.357303 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-be783b6c-86b4-4e09-abdb-91606e0dd8d1 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:45:54.358538 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-be783b6c-86b4-4e09-abdb-91606e0dd8d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:45:54.380708 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcee1ec1-9be7-4349-984e-cb0cf3696d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:54.381843 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcee1ec1-9be7-4349-984e-cb0cf3696d62 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:54.382411 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcee1ec1-9be7-4349-984e-cb0cf3696d62 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:54.382974 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcee1ec1-9be7-4349-984e-cb0cf3696d62 None admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:54.383958 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcee1ec1-9be7-4349-984e-cb0cf3696d62 None admin] RBAC: Adding query filter params (name=member) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:45:54.384582 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcee1ec1-9be7-4349-984e-cb0cf3696d62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:54.386495 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcee1ec1-9be7-4349-984e-cb0cf3696d62 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:54.398571 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 138/278] 10.222.0.64 () {56 vars in 1101 bytes} [Tue Aug 2 09:45:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:45:54.405574 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcee1ec1-9be7-4349-984e-cb0cf3696d62 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:54.438203 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:54.459306 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 141/279] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:45:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 09:45:54.478638 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be783b6c-86b4-4e09-abdb-91606e0dd8d1 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:54.531134 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcee1ec1-9be7-4349-984e-cb0cf3696d62 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:54.543528 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:54.581580 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61d948e-4bc1-42f5-8775-88158187a2ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:54.582092 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61d948e-4bc1-42f5-8775-88158187a2ca None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:54.582382 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61d948e-4bc1-42f5-8775-88158187a2ca None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:54.582660 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61d948e-4bc1-42f5-8775-88158187a2ca None admin] PATH_INFO: `/v3/users/e10a792b89354f1290e50e259936e4da` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:54.599840 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61d948e-4bc1-42f5-8775-88158187a2ca None admin] RBAC: Authorizing `identity:get_user(user_id=e10a792b89354f1290e50e259936e4da)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:54.601747 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61d948e-4bc1-42f5-8775-88158187a2ca None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:54.603681 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 139/280] 10.222.0.64 () {56 vars in 1177 bytes} [Tue Aug 2 09:45:54 2022] GET /identity/v3/users/e10a792b89354f1290e50e259936e4da => generated 293 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:45:54.609057 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be783b6c-86b4-4e09-abdb-91606e0dd8d1 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:54.609423 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61d948e-4bc1-42f5-8775-88158187a2ca None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:54.659365 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fb148a-025c-4bbe-a590-4602b0c4885e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:54.660519 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fb148a-025c-4bbe-a590-4602b0c4885e None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:54.661035 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fb148a-025c-4bbe-a590-4602b0c4885e None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:54.661546 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fb148a-025c-4bbe-a590-4602b0c4885e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:54.678450 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fb148a-025c-4bbe-a590-4602b0c4885e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 09:45:54.684868 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-31fb148a-025c-4bbe-a590-4602b0c4885e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 02 09:45:54.684868 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:45:54.684868 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:45:54.684868 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:45:54.684868 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:45:54.684868 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:45:54.684868 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:45:54.684868 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:45:54.684868 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:45:54.684868 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:45:54.684868 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:45:54.684868 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:45:54.684868 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 09:45:54.684868 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 09:45:54.684868 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 09:45:54.684868 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 09:45:54.685836 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 09:45:54.685836 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 09:45:54.685836 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 09:45:54.685836 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 09:45:54.685836 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:45:54.685836 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:45:54.685836 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:45:54.685836 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:45:54.685836 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:45:54.685836 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:45:54.685836 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:45:54.685836 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:45:54.685836 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:45:54.685836 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:45:54.685836 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:45:54.685836 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:45:54.686861 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:45:54.686861 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:45:54.686861 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:45:54.686861 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:45:54.686861 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 09:45:54.686861 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 09:45:54.686861 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 09:45:54.686861 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 09:45:54.686861 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 09:45:54.686861 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 09:45:54.686861 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 02 09:45:54.686861 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:45:54.686861 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 142/281] 10.222.0.64 () {56 vars in 1099 bytes} [Tue Aug 2 09:45:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 09:45:54.691183 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8252a430-113a-4e0e-867f-cb39da134712 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:54.691643 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8252a430-113a-4e0e-867f-cb39da134712 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:54.691929 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8252a430-113a-4e0e-867f-cb39da134712 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:54.693676 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 143/282] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:45:54 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:45:54.696904 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8252a430-113a-4e0e-867f-cb39da134712 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:54.744689 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61d948e-4bc1-42f5-8775-88158187a2ca None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:54.780531 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:54.792356 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9bbc8d-c7f3-46f1-8131-eefb3147c4a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:54.793638 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9bbc8d-c7f3-46f1-8131-eefb3147c4a8 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:54.794622 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9bbc8d-c7f3-46f1-8131-eefb3147c4a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:54.795233 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9bbc8d-c7f3-46f1-8131-eefb3147c4a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:54.796434 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9bbc8d-c7f3-46f1-8131-eefb3147c4a8 None admin] RBAC: Adding query filter params (role.id=66c1074780954f8e87ed11a5c4e8c36a, user.id=e10a792b89354f1290e50e259936e4da, scope.system=all) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:45:54.797034 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9bbc8d-c7f3-46f1-8131-eefb3147c4a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:54.799928 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9bbc8d-c7f3-46f1-8131-eefb3147c4a8 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:54.814284 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 140/283] 10.222.0.64 () {56 vars in 1308 bytes} [Tue Aug 2 09:45:54 2022] GET /identity/v3/role_assignments?user.id=e10a792b89354f1290e50e259936e4da&scope.system=all&role.id=66c1074780954f8e87ed11a5c4e8c36a => generated 226 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:45:54.816622 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f584eabc-90f8-4adc-a79a-e16bbce8490b None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:54.817135 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f584eabc-90f8-4adc-a79a-e16bbce8490b None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:54.818201 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f584eabc-90f8-4adc-a79a-e16bbce8490b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:54.843102 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-f584eabc-90f8-4adc-a79a-e16bbce8490b None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:45:54.845010 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-f584eabc-90f8-4adc-a79a-e16bbce8490b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:45:54.847608 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8252a430-113a-4e0e-867f-cb39da134712 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:54.880923 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:54.895720 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09078088-b8fd-4b70-8080-84304acfd78e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:54.896770 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09078088-b8fd-4b70-8080-84304acfd78e None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:54.897137 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09078088-b8fd-4b70-8080-84304acfd78e None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:54.897515 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09078088-b8fd-4b70-8080-84304acfd78e None admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:54.898366 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09078088-b8fd-4b70-8080-84304acfd78e None admin] RBAC: Adding query filter params (name=reader) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:45:54.898892 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09078088-b8fd-4b70-8080-84304acfd78e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:54.901000 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09078088-b8fd-4b70-8080-84304acfd78e None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:54.915230 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 144/284] 10.222.0.64 () {56 vars in 1101 bytes} [Tue Aug 2 09:45:54 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:45:54.916889 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09078088-b8fd-4b70-8080-84304acfd78e None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:54.945484 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 141/285] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:45:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 09:45:54.994170 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f584eabc-90f8-4adc-a79a-e16bbce8490b None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:55.045927 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09078088-b8fd-4b70-8080-84304acfd78e None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:55.067896 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:55.100433 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0caee0-8b7c-460c-b7ba-5dd5397a78df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:55.101958 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0caee0-8b7c-460c-b7ba-5dd5397a78df None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:55.102455 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0caee0-8b7c-460c-b7ba-5dd5397a78df None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:55.102923 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0caee0-8b7c-460c-b7ba-5dd5397a78df None admin] PATH_INFO: `/v3/users/ad730f60ea8e41dca4715c61f0050afa` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:55.104373 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0caee0-8b7c-460c-b7ba-5dd5397a78df None admin] RBAC: Authorizing `identity:get_user(user_id=ad730f60ea8e41dca4715c61f0050afa)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:55.110335 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0caee0-8b7c-460c-b7ba-5dd5397a78df None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:55.112231 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 145/286] 10.222.0.64 () {56 vars in 1177 bytes} [Tue Aug 2 09:45:54 2022] GET /identity/v3/users/ad730f60ea8e41dca4715c61f0050afa => generated 297 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:45:55.117593 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0caee0-8b7c-460c-b7ba-5dd5397a78df None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:55.196037 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f584eabc-90f8-4adc-a79a-e16bbce8490b None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:55.242443 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f30dd5b-b962-48fd-a91a-5b58978e80f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:55.243105 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f30dd5b-b962-48fd-a91a-5b58978e80f7 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:55.243395 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f30dd5b-b962-48fd-a91a-5b58978e80f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:55.243674 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f30dd5b-b962-48fd-a91a-5b58978e80f7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:55.268961 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f30dd5b-b962-48fd-a91a-5b58978e80f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 09:45:55.274230 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-4f30dd5b-b962-48fd-a91a-5b58978e80f7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 02 09:45:55.274230 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:45:55.274230 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:45:55.274230 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:45:55.274230 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:45:55.274230 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:45:55.274230 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:45:55.274230 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:45:55.274230 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:45:55.274230 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:45:55.274230 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:45:55.274230 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:45:55.274230 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 09:45:55.274230 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 09:45:55.274230 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 09:45:55.274230 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 09:45:55.275118 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 09:45:55.275118 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 09:45:55.275118 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 09:45:55.275118 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 09:45:55.275118 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:45:55.275118 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:45:55.275118 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:45:55.275118 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:45:55.275118 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:45:55.275118 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:45:55.275118 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:45:55.275118 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:45:55.275118 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:45:55.275118 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:45:55.275118 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:45:55.275118 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:45:55.275930 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:45:55.275930 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:45:55.275930 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:45:55.275930 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:45:55.275930 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 09:45:55.275930 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 09:45:55.275930 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 09:45:55.275930 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 09:45:55.275930 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 09:45:55.275930 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 09:45:55.275930 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 02 09:45:55.275930 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:45:55.283379 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 142/287] 10.222.0.64 () {56 vars in 1099 bytes} [Tue Aug 2 09:45:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:45:55.288054 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f30dd5b-b962-48fd-a91a-5b58978e80f7 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:55.320635 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:55.327033 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0caee0-8b7c-460c-b7ba-5dd5397a78df None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:55.390934 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf1e2cd-688a-4648-9f70-a0a869f21522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:55.391570 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf1e2cd-688a-4648-9f70-a0a869f21522 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:55.391862 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf1e2cd-688a-4648-9f70-a0a869f21522 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:55.392141 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf1e2cd-688a-4648-9f70-a0a869f21522 None admin] PATH_INFO: `/v3/projects/bc7fa79dde7e42ae92a2a42495efdcc2` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:55.394463 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf1e2cd-688a-4648-9f70-a0a869f21522 None admin] RBAC: Authorizing `identity:get_project(project_id=bc7fa79dde7e42ae92a2a42495efdcc2)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:55.395950 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf1e2cd-688a-4648-9f70-a0a869f21522 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:55.404024 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 146/288] 10.222.0.64 () {56 vars in 1186 bytes} [Tue Aug 2 09:45:55 2022] GET /identity/v3/projects/bc7fa79dde7e42ae92a2a42495efdcc2 => generated 302 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:45:55.407459 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf1e2cd-688a-4648-9f70-a0a869f21522 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:55.474455 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f30dd5b-b962-48fd-a91a-5b58978e80f7 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:55.524788 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12eb2105-b405-4479-a73c-2c02944a8d78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:55.526174 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eb2105-b405-4479-a73c-2c02944a8d78 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:55.526656 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eb2105-b405-4479-a73c-2c02944a8d78 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:55.527184 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eb2105-b405-4479-a73c-2c02944a8d78 None admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:55.528145 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12eb2105-b405-4479-a73c-2c02944a8d78 None admin] RBAC: Adding query filter params (name=member) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:45:55.528665 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12eb2105-b405-4479-a73c-2c02944a8d78 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:55.530269 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12eb2105-b405-4479-a73c-2c02944a8d78 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:55.532595 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:55.540928 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 143/289] 10.222.0.64 () {56 vars in 1101 bytes} [Tue Aug 2 09:45:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:45:55.546977 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12eb2105-b405-4479-a73c-2c02944a8d78 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:55.661199 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:55.675744 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf1e2cd-688a-4648-9f70-a0a869f21522 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:55.684578 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12eb2105-b405-4479-a73c-2c02944a8d78 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:55.730075 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c914e2-77e5-4404-a3c5-0a5faf285ddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:55.731277 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c914e2-77e5-4404-a3c5-0a5faf285ddd None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:55.731871 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c914e2-77e5-4404-a3c5-0a5faf285ddd None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:55.732574 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c914e2-77e5-4404-a3c5-0a5faf285ddd None admin] PATH_INFO: `/v3/users/31178481748442ada86cab1c8bc75b18` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:55.735124 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c914e2-77e5-4404-a3c5-0a5faf285ddd None admin] RBAC: Authorizing `identity:get_user(user_id=31178481748442ada86cab1c8bc75b18)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:55.736778 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c914e2-77e5-4404-a3c5-0a5faf285ddd None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:55.739083 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 144/290] 10.222.0.64 () {56 vars in 1177 bytes} [Tue Aug 2 09:45:55 2022] GET /identity/v3/users/31178481748442ada86cab1c8bc75b18 => generated 297 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:45:55.745511 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c914e2-77e5-4404-a3c5-0a5faf285ddd None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:55.787969 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fd1934-604a-4d18-b495-8687ce4f53c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:55.788770 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fd1934-604a-4d18-b495-8687ce4f53c2 None admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:55.789055 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fd1934-604a-4d18-b495-8687ce4f53c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:55.789319 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fd1934-604a-4d18-b495-8687ce4f53c2 None admin] PATH_INFO: `/v3/projects/bc7fa79dde7e42ae92a2a42495efdcc2/users/ad730f60ea8e41dca4715c61f0050afa/roles/7fce0045dfa049bdb7ae0d2424de9f32` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:55.795648 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fd1934-604a-4d18-b495-8687ce4f53c2 None admin] RBAC: Authorizing `identity:create_grant(project_id=bc7fa79dde7e42ae92a2a42495efdcc2, user_id=ad730f60ea8e41dca4715c61f0050afa, role_id=7fce0045dfa049bdb7ae0d2424de9f32)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:55.797139 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fd1934-604a-4d18-b495-8687ce4f53c2 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:55.818800 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 147/291] 10.222.0.64 () {58 vars in 1439 bytes} [Tue Aug 2 09:45:55 2022] PUT /identity/v3/projects/bc7fa79dde7e42ae92a2a42495efdcc2/users/ad730f60ea8e41dca4715c61f0050afa/roles/7fce0045dfa049bdb7ae0d2424de9f32 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:45:55.905941 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c914e2-77e5-4404-a3c5-0a5faf285ddd None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:55.954753 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1d03c0-de48-4d54-9915-1345761d61eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:55.955360 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1d03c0-de48-4d54-9915-1345761d61eb None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:55.955627 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1d03c0-de48-4d54-9915-1345761d61eb None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:55.955884 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1d03c0-de48-4d54-9915-1345761d61eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:55.984015 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:56.060425 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1d03c0-de48-4d54-9915-1345761d61eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:56.061927 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1d03c0-de48-4d54-9915-1345761d61eb None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:56.215291 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 145/292] 10.222.0.64 () {58 vars in 1282 bytes} [Tue Aug 2 09:45:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 09:45:56.222221 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fd1934-604a-4d18-b495-8687ce4f53c2 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:56.360987 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fd1934-604a-4d18-b495-8687ce4f53c2 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:56.373054 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:56.405459 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716bddd0-e452-4641-954f-6944855d9d9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:56.406106 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716bddd0-e452-4641-954f-6944855d9d9b None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:56.406388 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716bddd0-e452-4641-954f-6944855d9d9b None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:56.406663 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716bddd0-e452-4641-954f-6944855d9d9b None admin] PATH_INFO: `/v3/projects/bc7fa79dde7e42ae92a2a42495efdcc2` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:56.407828 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716bddd0-e452-4641-954f-6944855d9d9b None admin] RBAC: Authorizing `identity:get_project(project_id=bc7fa79dde7e42ae92a2a42495efdcc2)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:56.409546 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716bddd0-e452-4641-954f-6944855d9d9b None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:56.411320 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 148/293] 10.222.0.64 () {56 vars in 1186 bytes} [Tue Aug 2 09:45:56 2022] GET /identity/v3/projects/bc7fa79dde7e42ae92a2a42495efdcc2 => generated 302 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:45:56.417456 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1d03c0-de48-4d54-9915-1345761d61eb None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:56.471074 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:56.561853 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1d03c0-de48-4d54-9915-1345761d61eb None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:56.615872 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7a8a84-c92b-4bd3-9def-0a88ec6d942d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:56.616810 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7a8a84-c92b-4bd3-9def-0a88ec6d942d None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:56.617096 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7a8a84-c92b-4bd3-9def-0a88ec6d942d None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:56.617372 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7a8a84-c92b-4bd3-9def-0a88ec6d942d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:56.618189 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7a8a84-c92b-4bd3-9def-0a88ec6d942d None admin] RBAC: Adding query filter params (role.id=66c1074780954f8e87ed11a5c4e8c36a, scope.project.id=bc7fa79dde7e42ae92a2a42495efdcc2, user.id=31178481748442ada86cab1c8bc75b18) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:45:56.618499 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7a8a84-c92b-4bd3-9def-0a88ec6d942d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:56.620146 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7a8a84-c92b-4bd3-9def-0a88ec6d942d None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:56.629106 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 146/294] 10.222.0.64 () {56 vars in 1374 bytes} [Tue Aug 2 09:45:56 2022] GET /identity/v3/role_assignments?user.id=31178481748442ada86cab1c8bc75b18&scope.project.id=bc7fa79dde7e42ae92a2a42495efdcc2&role.id=66c1074780954f8e87ed11a5c4e8c36a => generated 608 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:45:56.947165 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc5905c-de6a-4425-9f6f-4e5b0a91bc55 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:56.947491 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc5905c-de6a-4425-9f6f-4e5b0a91bc55 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:56.947729 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc5905c-de6a-4425-9f6f-4e5b0a91bc55 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:56.949448 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 149/295] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:45:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:45:56.954914 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b011297-6596-4660-9e53-bee1042368eb None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:56.955404 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b011297-6596-4660-9e53-bee1042368eb None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:56.955487 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b011297-6596-4660-9e53-bee1042368eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:56.980585 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-6b011297-6596-4660-9e53-bee1042368eb None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:45:56.981803 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-6b011297-6596-4660-9e53-bee1042368eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:45:57.074614 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 147/296] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:45:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 09:45:57.100152 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc5905c-de6a-4425-9f6f-4e5b0a91bc55 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:57.219670 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc5905c-de6a-4425-9f6f-4e5b0a91bc55 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:57.256538 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7995b379-a176-439f-a5ae-220cd0cac650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:57.257171 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7995b379-a176-439f-a5ae-220cd0cac650 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:57.257465 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7995b379-a176-439f-a5ae-220cd0cac650 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:57.257743 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7995b379-a176-439f-a5ae-220cd0cac650 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:57.268967 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7995b379-a176-439f-a5ae-220cd0cac650 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 09:45:57.274688 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-7995b379-a176-439f-a5ae-220cd0cac650 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 02 09:45:57.274688 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:45:57.274688 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:45:57.274688 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:45:57.274688 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:45:57.274688 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:45:57.274688 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:45:57.274688 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:45:57.274688 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:45:57.274688 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:45:57.274688 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:45:57.274688 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:45:57.274688 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 09:45:57.274688 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 09:45:57.274688 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 09:45:57.274688 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 09:45:57.275636 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 09:45:57.275636 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 09:45:57.275636 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 09:45:57.275636 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 09:45:57.275636 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:45:57.275636 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:45:57.275636 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:45:57.275636 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:45:57.275636 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:45:57.275636 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:45:57.275636 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:45:57.275636 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:45:57.275636 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:45:57.275636 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:45:57.275636 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:45:57.275636 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:45:57.276459 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:45:57.276459 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:45:57.276459 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:45:57.276459 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:45:57.276459 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 09:45:57.276459 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 09:45:57.276459 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 09:45:57.276459 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 09:45:57.276459 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 09:45:57.276459 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 09:45:57.276459 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 02 09:45:57.276459 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:45:57.277089 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce91b38a-4aa6-4d21-9d0e-058c61236854 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:57.277089 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce91b38a-4aa6-4d21-9d0e-058c61236854 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:57.277089 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce91b38a-4aa6-4d21-9d0e-058c61236854 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:57.278335 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 148/298] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:45:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:45:57.278460 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 150/298] 10.222.0.64 () {56 vars in 1099 bytes} [Tue Aug 2 09:45:57 2022] GET /identity/v3/roles/reader => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:45:57.281887 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce91b38a-4aa6-4d21-9d0e-058c61236854 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:57.284136 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6978e03f-eb1b-4215-b766-1aba6379c520 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:57.284631 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6978e03f-eb1b-4215-b766-1aba6379c520 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:57.285106 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6978e03f-eb1b-4215-b766-1aba6379c520 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:57.308563 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-6978e03f-eb1b-4215-b766-1aba6379c520 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:45:57.310364 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-6978e03f-eb1b-4215-b766-1aba6379c520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:45:57.346800 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:57.361647 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:57.416323 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 151/299] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:45:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 09:45:57.417873 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ffad02-7de6-43e7-bb64-849958ca1cf8 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:57.418164 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ffad02-7de6-43e7-bb64-849958ca1cf8 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:57.418524 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ffad02-7de6-43e7-bb64-849958ca1cf8 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:57.419944 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 152/300] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:45:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:45:57.425404 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c96edc-706f-43c7-8f59-902ce96e639f None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:57.425654 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c96edc-706f-43c7-8f59-902ce96e639f None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:57.425914 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c96edc-706f-43c7-8f59-902ce96e639f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:57.432452 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce91b38a-4aa6-4d21-9d0e-058c61236854 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:57.449262 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-43c96edc-706f-43c7-8f59-902ce96e639f None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:45:57.450474 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-43c96edc-706f-43c7-8f59-902ce96e639f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:45:57.471197 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb0d3a9-446c-4e24-ad71-0e504a156b0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:57.471828 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb0d3a9-446c-4e24-ad71-0e504a156b0e None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:57.472122 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb0d3a9-446c-4e24-ad71-0e504a156b0e None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:57.472442 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb0d3a9-446c-4e24-ad71-0e504a156b0e None admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:57.473672 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb0d3a9-446c-4e24-ad71-0e504a156b0e None admin] RBAC: Adding query filter params (name=reader) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:45:57.474197 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb0d3a9-446c-4e24-ad71-0e504a156b0e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:57.477090 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb0d3a9-446c-4e24-ad71-0e504a156b0e None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:57.486384 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 149/301] 10.222.0.64 () {56 vars in 1101 bytes} [Tue Aug 2 09:45:57 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:45:57.487244 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb0d3a9-446c-4e24-ad71-0e504a156b0e None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:57.539924 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 153/302] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:45:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 09:45:57.540643 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c96edc-706f-43c7-8f59-902ce96e639f None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:57.624831 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb0d3a9-446c-4e24-ad71-0e504a156b0e None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:57.633656 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:57.660338 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:57.664664 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c96edc-706f-43c7-8f59-902ce96e639f None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:57.665362 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe5cfd7-9ab4-418b-996a-f9dc2b494e4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:57.666019 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe5cfd7-9ab4-418b-996a-f9dc2b494e4d None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:57.666309 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe5cfd7-9ab4-418b-996a-f9dc2b494e4d None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:57.666643 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe5cfd7-9ab4-418b-996a-f9dc2b494e4d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:57.668094 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe5cfd7-9ab4-418b-996a-f9dc2b494e4d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:57.669352 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe5cfd7-9ab4-418b-996a-f9dc2b494e4d None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:57.676289 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 150/303] 10.222.0.64 () {56 vars in 1108 bytes} [Tue Aug 2 09:45:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:45:57.676289 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe5cfd7-9ab4-418b-996a-f9dc2b494e4d None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:57.706931 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4871ec2f-33a3-4e60-ac6d-672e49c219ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:57.707584 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4871ec2f-33a3-4e60-ac6d-672e49c219ea None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:57.707868 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4871ec2f-33a3-4e60-ac6d-672e49c219ea None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:57.708149 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4871ec2f-33a3-4e60-ac6d-672e49c219ea None admin] PATH_INFO: `/v3/users/cc0f20e1976e4ef4843f697366ea0730` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:57.722301 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4871ec2f-33a3-4e60-ac6d-672e49c219ea None admin] RBAC: Authorizing `identity:get_user(user_id=cc0f20e1976e4ef4843f697366ea0730)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:57.723775 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4871ec2f-33a3-4e60-ac6d-672e49c219ea None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:57.725741 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 154/304] 10.222.0.64 () {56 vars in 1177 bytes} [Tue Aug 2 09:45:57 2022] GET /identity/v3/users/cc0f20e1976e4ef4843f697366ea0730 => generated 293 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:45:57.731386 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4871ec2f-33a3-4e60-ac6d-672e49c219ea None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:57.795587 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe5cfd7-9ab4-418b-996a-f9dc2b494e4d None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:57.833109 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5b8880-c1db-4617-8723-41644be9bb5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:57.833757 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5b8880-c1db-4617-8723-41644be9bb5e None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:57.834041 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5b8880-c1db-4617-8723-41644be9bb5e None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:57.834317 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5b8880-c1db-4617-8723-41644be9bb5e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:57.845614 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5b8880-c1db-4617-8723-41644be9bb5e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 09:45:57.852278 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-ce5b8880-c1db-4617-8723-41644be9bb5e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 02 09:45:57.852278 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:45:57.852278 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:45:57.852278 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:45:57.852278 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:45:57.852278 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:45:57.852278 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:45:57.852278 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:45:57.852278 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:45:57.852278 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:45:57.852278 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:45:57.852278 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:45:57.852278 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 09:45:57.852278 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 09:45:57.852278 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 09:45:57.852278 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 09:45:57.853366 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 09:45:57.853366 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 09:45:57.853366 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 09:45:57.853366 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 09:45:57.853366 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:45:57.853366 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:45:57.853366 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:45:57.853366 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:45:57.853366 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:45:57.853366 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:45:57.853366 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:45:57.853366 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:45:57.853366 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:45:57.853366 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:45:57.853366 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:45:57.853366 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:45:57.854429 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:45:57.854429 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:45:57.854429 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:45:57.854429 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:45:57.854429 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 09:45:57.854429 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 09:45:57.854429 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 09:45:57.854429 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 09:45:57.854429 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 09:45:57.854429 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 09:45:57.854429 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 02 09:45:57.854429 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:45:57.854429 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 151/305] 10.222.0.64 () {56 vars in 1099 bytes} [Tue Aug 2 09:45:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:45:57.854429 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5b8880-c1db-4617-8723-41644be9bb5e None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:57.867856 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4871ec2f-33a3-4e60-ac6d-672e49c219ea None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:57.902020 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:57.914671 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2535166-9d52-41ab-9144-eb5510d867fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:57.915299 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2535166-9d52-41ab-9144-eb5510d867fd None admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:57.915580 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2535166-9d52-41ab-9144-eb5510d867fd None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:57.915856 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2535166-9d52-41ab-9144-eb5510d867fd None admin] PATH_INFO: `/v3/groups` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:57.916566 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2535166-9d52-41ab-9144-eb5510d867fd None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:57.918074 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2535166-9d52-41ab-9144-eb5510d867fd None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:57.937752 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 155/306] 10.222.0.64 () {60 vars in 1134 bytes} [Tue Aug 2 09:45:57 2022] POST /identity/v3/groups => generated 229 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:45:57.938599 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2535166-9d52-41ab-9144-eb5510d867fd None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:57.980153 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:57.996915 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5b8880-c1db-4617-8723-41644be9bb5e None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:58.066416 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238db362-6f0c-40fd-946c-47d0c9cb4904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:58.067848 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238db362-6f0c-40fd-946c-47d0c9cb4904 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:58.068595 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238db362-6f0c-40fd-946c-47d0c9cb4904 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:58.069327 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238db362-6f0c-40fd-946c-47d0c9cb4904 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:58.071098 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238db362-6f0c-40fd-946c-47d0c9cb4904 None admin] RBAC: Adding query filter params (role.id=7fce0045dfa049bdb7ae0d2424de9f32, user.id=cc0f20e1976e4ef4843f697366ea0730, scope.system=all) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:45:58.071951 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238db362-6f0c-40fd-946c-47d0c9cb4904 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:58.074827 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238db362-6f0c-40fd-946c-47d0c9cb4904 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:58.083612 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 152/307] 10.222.0.64 () {56 vars in 1308 bytes} [Tue Aug 2 09:45:57 2022] GET /identity/v3/role_assignments?user.id=cc0f20e1976e4ef4843f697366ea0730&scope.system=all&role.id=7fce0045dfa049bdb7ae0d2424de9f32 => generated 226 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:45:58.127691 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2535166-9d52-41ab-9144-eb5510d867fd None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:58.170561 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c0e1b7-9436-487c-b9a8-b5ab35fd81ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:58.171216 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c0e1b7-9436-487c-b9a8-b5ab35fd81ed None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:58.171501 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c0e1b7-9436-487c-b9a8-b5ab35fd81ed None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:58.171776 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c0e1b7-9436-487c-b9a8-b5ab35fd81ed None admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:58.172566 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c0e1b7-9436-487c-b9a8-b5ab35fd81ed None admin] RBAC: Adding query filter params (name=member) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:45:58.172873 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c0e1b7-9436-487c-b9a8-b5ab35fd81ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:58.175069 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c0e1b7-9436-487c-b9a8-b5ab35fd81ed None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:58.185547 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 156/308] 10.222.0.64 () {56 vars in 1101 bytes} [Tue Aug 2 09:45:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:45:58.191855 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238db362-6f0c-40fd-946c-47d0c9cb4904 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:58.319806 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:58.324448 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238db362-6f0c-40fd-946c-47d0c9cb4904 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:58.375373 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399c7b16-6a34-4af0-b743-743d13286eae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:58.376036 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399c7b16-6a34-4af0-b743-743d13286eae None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:58.376350 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399c7b16-6a34-4af0-b743-743d13286eae None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:58.376627 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399c7b16-6a34-4af0-b743-743d13286eae None admin] PATH_INFO: `/v3/users/e10a792b89354f1290e50e259936e4da` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:58.378424 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399c7b16-6a34-4af0-b743-743d13286eae None admin] RBAC: Authorizing `identity:get_user(user_id=e10a792b89354f1290e50e259936e4da)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:58.379893 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399c7b16-6a34-4af0-b743-743d13286eae None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:58.381913 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 153/309] 10.222.0.64 () {56 vars in 1177 bytes} [Tue Aug 2 09:45:58 2022] GET /identity/v3/users/e10a792b89354f1290e50e259936e4da => generated 293 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:45:58.387743 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c0e1b7-9436-487c-b9a8-b5ab35fd81ed None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:58.458560 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aaa82d-45bd-4600-bb37-d3a2b77f4311 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:58.459271 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aaa82d-45bd-4600-bb37-d3a2b77f4311 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:58.459929 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aaa82d-45bd-4600-bb37-d3a2b77f4311 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:58.461842 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 154/310] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:45:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:45:58.468789 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c33445-dfa8-4a10-a49e-39ef1d99f0a4 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:58.469089 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c33445-dfa8-4a10-a49e-39ef1d99f0a4 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:58.469342 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c33445-dfa8-4a10-a49e-39ef1d99f0a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:58.493563 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-24c33445-dfa8-4a10-a49e-39ef1d99f0a4 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:45:58.494794 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-24c33445-dfa8-4a10-a49e-39ef1d99f0a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:45:58.506780 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:58.543284 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c0e1b7-9436-487c-b9a8-b5ab35fd81ed None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:58.581340 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aa9155-a684-4363-903e-8877ad35cdf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:58.581927 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aa9155-a684-4363-903e-8877ad35cdf0 None admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:58.582210 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aa9155-a684-4363-903e-8877ad35cdf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:58.582490 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aa9155-a684-4363-903e-8877ad35cdf0 None admin] PATH_INFO: `/v3/system/users/e10a792b89354f1290e50e259936e4da/roles/66c1074780954f8e87ed11a5c4e8c36a` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:58.583986 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88aa9155-a684-4363-903e-8877ad35cdf0 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=e10a792b89354f1290e50e259936e4da, role_id=66c1074780954f8e87ed11a5c4e8c36a)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:58.585446 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88aa9155-a684-4363-903e-8877ad35cdf0 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:58.596273 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 157/311] 10.222.0.64 () {58 vars in 1334 bytes} [Tue Aug 2 09:45:58 2022] PUT /identity/v3/system/users/e10a792b89354f1290e50e259936e4da/roles/66c1074780954f8e87ed11a5c4e8c36a => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:45:58.610034 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 155/312] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:45:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 09:45:58.633807 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aa9155-a684-4363-903e-8877ad35cdf0 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:58.780715 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aa9155-a684-4363-903e-8877ad35cdf0 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:58.820815 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d612a83a-1fe8-45a4-a17d-5fa23344891e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:58.821478 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d612a83a-1fe8-45a4-a17d-5fa23344891e None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:58.821763 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d612a83a-1fe8-45a4-a17d-5fa23344891e None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:58.822080 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d612a83a-1fe8-45a4-a17d-5fa23344891e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:58.834893 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d612a83a-1fe8-45a4-a17d-5fa23344891e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 09:45:58.840926 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-d612a83a-1fe8-45a4-a17d-5fa23344891e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 02 09:45:58.840926 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:45:58.840926 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:45:58.840926 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:45:58.840926 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:45:58.840926 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:45:58.840926 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:45:58.840926 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:45:58.840926 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:45:58.840926 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:45:58.840926 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:45:58.840926 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:45:58.840926 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 09:45:58.840926 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 09:45:58.840926 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 09:45:58.840926 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 09:45:58.841893 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 09:45:58.841893 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 09:45:58.841893 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 09:45:58.841893 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 09:45:58.841893 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:45:58.841893 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:45:58.841893 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:45:58.841893 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:45:58.841893 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:45:58.841893 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:45:58.841893 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:45:58.841893 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:45:58.841893 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:45:58.841893 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:45:58.841893 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:45:58.841893 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:45:58.842701 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:45:58.842701 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:45:58.842701 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:45:58.842701 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:45:58.842701 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 09:45:58.842701 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 09:45:58.842701 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 09:45:58.842701 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 09:45:58.842701 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 09:45:58.842701 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 09:45:58.842701 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 02 09:45:58.842701 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:45:58.843315 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 158/313] 10.222.0.64 () {56 vars in 1099 bytes} [Tue Aug 2 09:45:58 2022] GET /identity/v3/roles/reader => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:45:58.849135 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c33445-dfa8-4a10-a49e-39ef1d99f0a4 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:58.956621 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:58.962857 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c33445-dfa8-4a10-a49e-39ef1d99f0a4 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:59.000927 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2fbb3a-0c92-404d-8ba6-5227e4c752d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:59.001833 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2fbb3a-0c92-404d-8ba6-5227e4c752d2 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:59.001833 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2fbb3a-0c92-404d-8ba6-5227e4c752d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:59.002160 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2fbb3a-0c92-404d-8ba6-5227e4c752d2 None admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:59.003354 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2fbb3a-0c92-404d-8ba6-5227e4c752d2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:45:59.003660 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2fbb3a-0c92-404d-8ba6-5227e4c752d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:59.005074 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2fbb3a-0c92-404d-8ba6-5227e4c752d2 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:59.014717 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 156/314] 10.222.0.64 () {56 vars in 1101 bytes} [Tue Aug 2 09:45:58 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:45:59.021399 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d612a83a-1fe8-45a4-a17d-5fa23344891e None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:59.025669 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:59.145163 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d612a83a-1fe8-45a4-a17d-5fa23344891e None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:59.182512 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4eb288-ed3d-4f84-abd2-a0adee545b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:59.183533 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4eb288-ed3d-4f84-abd2-a0adee545b8d None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:59.184011 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4eb288-ed3d-4f84-abd2-a0adee545b8d None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:59.184509 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4eb288-ed3d-4f84-abd2-a0adee545b8d None admin] PATH_INFO: `/v3/users/ad730f60ea8e41dca4715c61f0050afa` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:59.185909 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4eb288-ed3d-4f84-abd2-a0adee545b8d None admin] RBAC: Authorizing `identity:get_user(user_id=ad730f60ea8e41dca4715c61f0050afa)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:59.187605 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4eb288-ed3d-4f84-abd2-a0adee545b8d None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:59.189516 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 159/315] 10.222.0.64 () {56 vars in 1177 bytes} [Tue Aug 2 09:45:59 2022] GET /identity/v3/users/ad730f60ea8e41dca4715c61f0050afa => generated 297 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:45:59.196286 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2fbb3a-0c92-404d-8ba6-5227e4c752d2 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:59.307226 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2fbb3a-0c92-404d-8ba6-5227e4c752d2 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:59.343754 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b22920f-8b12-4367-9914-a0519853c8e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:59.344424 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b22920f-8b12-4367-9914-a0519853c8e6 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:59.344716 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b22920f-8b12-4367-9914-a0519853c8e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:59.345001 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b22920f-8b12-4367-9914-a0519853c8e6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:59.384783 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:59.429360 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b22920f-8b12-4367-9914-a0519853c8e6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:59.430746 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b22920f-8b12-4367-9914-a0519853c8e6 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:59.536515 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 157/316] 10.222.0.64 () {58 vars in 1282 bytes} [Tue Aug 2 09:45:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 09:45:59.541973 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4eb288-ed3d-4f84-abd2-a0adee545b8d None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:59.633424 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:59.668842 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4eb288-ed3d-4f84-abd2-a0adee545b8d None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:59.720899 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d3beae-f3a4-40af-91e0-cc593cbb3fb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:59.721845 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d3beae-f3a4-40af-91e0-cc593cbb3fb1 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:59.722281 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d3beae-f3a4-40af-91e0-cc593cbb3fb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:59.722719 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d3beae-f3a4-40af-91e0-cc593cbb3fb1 None admin] PATH_INFO: `/v3/projects/bc7fa79dde7e42ae92a2a42495efdcc2` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:59.724646 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d3beae-f3a4-40af-91e0-cc593cbb3fb1 None admin] RBAC: Authorizing `identity:get_project(project_id=bc7fa79dde7e42ae92a2a42495efdcc2)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:59.727505 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d3beae-f3a4-40af-91e0-cc593cbb3fb1 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:59.730598 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 160/317] 10.222.0.64 () {56 vars in 1186 bytes} [Tue Aug 2 09:45:59 2022] GET /identity/v3/projects/bc7fa79dde7e42ae92a2a42495efdcc2 => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:45:59.736494 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b22920f-8b12-4367-9914-a0519853c8e6 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:45:59.804707 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:45:59.851942 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b22920f-8b12-4367-9914-a0519853c8e6 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:45:59.891128 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140e43f2-e997-498c-b16a-875cae125e5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:45:59.892137 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140e43f2-e997-498c-b16a-875cae125e5c None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:45:59.892700 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140e43f2-e997-498c-b16a-875cae125e5c None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:45:59.893232 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140e43f2-e997-498c-b16a-875cae125e5c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:45:59.894756 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140e43f2-e997-498c-b16a-875cae125e5c None admin] RBAC: Adding query filter params (role.id=7fce0045dfa049bdb7ae0d2424de9f32, scope.project.id=bc7fa79dde7e42ae92a2a42495efdcc2, user.id=ad730f60ea8e41dca4715c61f0050afa) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:45:59.895278 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140e43f2-e997-498c-b16a-875cae125e5c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:45:59.897029 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140e43f2-e997-498c-b16a-875cae125e5c None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:45:59.904388 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 158/318] 10.222.0.64 () {56 vars in 1374 bytes} [Tue Aug 2 09:45:59 2022] GET /identity/v3/role_assignments?user.id=ad730f60ea8e41dca4715c61f0050afa&scope.project.id=bc7fa79dde7e42ae92a2a42495efdcc2&role.id=7fce0045dfa049bdb7ae0d2424de9f32 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:00.203561 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe11756-1675-440e-afd6-013efa2b55f6 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:00.203846 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe11756-1675-440e-afd6-013efa2b55f6 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:00.204329 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe11756-1675-440e-afd6-013efa2b55f6 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:00.205720 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 161/319] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:46:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:46:00.211146 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb00703d-3e9a-4b20-9174-7dc846db6b39 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:00.211629 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb00703d-3e9a-4b20-9174-7dc846db6b39 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:00.211705 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb00703d-3e9a-4b20-9174-7dc846db6b39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:00.232015 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-fb00703d-3e9a-4b20-9174-7dc846db6b39 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:46:00.233277 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-fb00703d-3e9a-4b20-9174-7dc846db6b39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:46:00.313976 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 159/320] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:46:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 09:46:00.337393 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe11756-1675-440e-afd6-013efa2b55f6 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:00.385429 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ea88b7-eda8-4e16-8f6b-ea3527a2518d None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:00.385763 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ea88b7-eda8-4e16-8f6b-ea3527a2518d None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:00.385996 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ea88b7-eda8-4e16-8f6b-ea3527a2518d None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:00.387378 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 160/321] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:46:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:46:00.392704 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba0b9df-121c-4a70-b02f-13616d35aceb None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:00.393000 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba0b9df-121c-4a70-b02f-13616d35aceb None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:00.393279 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba0b9df-121c-4a70-b02f-13616d35aceb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:00.414895 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-8ba0b9df-121c-4a70-b02f-13616d35aceb None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:46:00.416290 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-8ba0b9df-121c-4a70-b02f-13616d35aceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:46:00.453756 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe11756-1675-440e-afd6-013efa2b55f6 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:00.476074 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:00.493040 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18bd326-8f4a-44f4-938d-4ef57b3fde66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:00.493717 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18bd326-8f4a-44f4-938d-4ef57b3fde66 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:00.493993 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18bd326-8f4a-44f4-938d-4ef57b3fde66 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:00.494275 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18bd326-8f4a-44f4-938d-4ef57b3fde66 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:00.495454 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18bd326-8f4a-44f4-938d-4ef57b3fde66 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:00.496955 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18bd326-8f4a-44f4-938d-4ef57b3fde66 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:00.498819 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 162/322] 10.222.0.64 () {56 vars in 1108 bytes} [Tue Aug 2 09:46:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:00.503033 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 161/323] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:46:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 09:46:00.505526 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18bd326-8f4a-44f4-938d-4ef57b3fde66 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:00.515570 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:00.525138 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba0b9df-121c-4a70-b02f-13616d35aceb None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:00.629368 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18bd326-8f4a-44f4-938d-4ef57b3fde66 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:00.661743 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba0b9df-121c-4a70-b02f-13616d35aceb None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:00.674169 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1b39a5-18cd-4a25-b2bc-f4643c3040b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:00.674813 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1b39a5-18cd-4a25-b2bc-f4643c3040b2 None admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:00.675102 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1b39a5-18cd-4a25-b2bc-f4643c3040b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:00.675382 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1b39a5-18cd-4a25-b2bc-f4643c3040b2 None admin] PATH_INFO: `/v3/groups` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:00.676072 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1b39a5-18cd-4a25-b2bc-f4643c3040b2 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:00.677585 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1b39a5-18cd-4a25-b2bc-f4643c3040b2 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:00.708772 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 163/324] 10.222.0.64 () {60 vars in 1134 bytes} [Tue Aug 2 09:46:00 2022] POST /identity/v3/groups => generated 226 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 09:46:00.708952 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f784fa0-b697-46cd-8229-a5ff2344858b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:00.708952 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f784fa0-b697-46cd-8229-a5ff2344858b None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:00.709303 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f784fa0-b697-46cd-8229-a5ff2344858b None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:00.709630 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f784fa0-b697-46cd-8229-a5ff2344858b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:00.724134 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f784fa0-b697-46cd-8229-a5ff2344858b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 09:46:00.730257 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-6f784fa0-b697-46cd-8229-a5ff2344858b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 02 09:46:00.730257 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:00.730257 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:00.730257 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:00.730257 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:00.730257 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:00.730257 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:00.730257 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:00.730257 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:00.730257 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:00.730257 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:00.730257 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:00.730257 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 09:46:00.730257 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 09:46:00.730257 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 09:46:00.730257 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 09:46:00.731255 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 09:46:00.731255 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 09:46:00.731255 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 09:46:00.731255 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 09:46:00.731255 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:00.731255 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:00.731255 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:00.731255 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:00.731255 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:00.731255 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:00.731255 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:00.731255 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:00.731255 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:00.731255 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:00.731255 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:00.731255 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:00.737544 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:00.737544 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:00.737544 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:00.737544 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:00.737544 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 09:46:00.737544 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 09:46:00.737544 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 09:46:00.737544 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 09:46:00.737544 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 09:46:00.737544 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 09:46:00.737544 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 02 09:46:00.737544 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:46:00.737544 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 162/325] 10.222.0.64 () {56 vars in 1099 bytes} [Tue Aug 2 09:46:00 2022] GET /identity/v3/roles/reader => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:46:00.740019 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1b39a5-18cd-4a25-b2bc-f4643c3040b2 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:00.802239 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a69200-14ea-48b2-8c5c-1ed22302396c None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:00.802605 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a69200-14ea-48b2-8c5c-1ed22302396c None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:00.802824 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a69200-14ea-48b2-8c5c-1ed22302396c None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:00.804469 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 163/326] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:46:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 09:46:00.810018 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f60118d-af00-40cd-a3b1-96903a36c233 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:00.810335 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f60118d-af00-40cd-a3b1-96903a36c233 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:00.810585 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f60118d-af00-40cd-a3b1-96903a36c233 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:00.830268 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:00.833504 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-8f60118d-af00-40cd-a3b1-96903a36c233 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:46:00.834835 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-8f60118d-af00-40cd-a3b1-96903a36c233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:46:00.881077 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1b39a5-18cd-4a25-b2bc-f4643c3040b2 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:00.895058 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:00.936119 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 164/327] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:46:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 09:46:00.944987 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9a5935-1adf-49b8-b7a0-15e0808d1738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:00.945931 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9a5935-1adf-49b8-b7a0-15e0808d1738 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:00.946223 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9a5935-1adf-49b8-b7a0-15e0808d1738 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:00.946505 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9a5935-1adf-49b8-b7a0-15e0808d1738 None admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:00.947242 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9a5935-1adf-49b8-b7a0-15e0808d1738 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:00.947550 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9a5935-1adf-49b8-b7a0-15e0808d1738 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:00.949026 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9a5935-1adf-49b8-b7a0-15e0808d1738 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:00.958972 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f60118d-af00-40cd-a3b1-96903a36c233 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:00.961683 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 164/328] 10.222.0.64 () {56 vars in 1101 bytes} [Tue Aug 2 09:46:00 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:00.967101 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9a5935-1adf-49b8-b7a0-15e0808d1738 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:01.093052 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f60118d-af00-40cd-a3b1-96903a36c233 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:01.105863 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9a5935-1adf-49b8-b7a0-15e0808d1738 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:01.146450 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896f949a-ea85-4b3c-bf95-15210721e0e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:01.147110 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896f949a-ea85-4b3c-bf95-15210721e0e3 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:01.147401 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896f949a-ea85-4b3c-bf95-15210721e0e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:01.147736 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896f949a-ea85-4b3c-bf95-15210721e0e3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:01.149429 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:01.162065 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e61684-11b2-4d0a-935f-ceb49dd54db6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:01.162739 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e61684-11b2-4d0a-935f-ceb49dd54db6 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:01.163170 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e61684-11b2-4d0a-935f-ceb49dd54db6 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:01.163365 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e61684-11b2-4d0a-935f-ceb49dd54db6 None admin] PATH_INFO: `/v3/users/cc0f20e1976e4ef4843f697366ea0730` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:01.165822 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e61684-11b2-4d0a-935f-ceb49dd54db6 None admin] RBAC: Authorizing `identity:get_user(user_id=cc0f20e1976e4ef4843f697366ea0730)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:01.165959 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896f949a-ea85-4b3c-bf95-15210721e0e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 09:46:01.167323 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e61684-11b2-4d0a-935f-ceb49dd54db6 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:01.171116 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 165/329] 10.222.0.64 () {56 vars in 1177 bytes} [Tue Aug 2 09:46:00 2022] GET /identity/v3/users/cc0f20e1976e4ef4843f697366ea0730 => generated 293 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:46:01.172449 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-896f949a-ea85-4b3c-bf95-15210721e0e3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 02 09:46:01.172449 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:01.172449 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:01.172449 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:01.172449 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:01.172449 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:01.172449 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:01.172449 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:01.172449 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:01.172449 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:01.172449 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:01.172449 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:01.172449 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 09:46:01.172449 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 09:46:01.172449 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 09:46:01.172449 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 09:46:01.173336 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 09:46:01.173336 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 09:46:01.173336 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 09:46:01.173336 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 09:46:01.173336 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:01.173336 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:01.173336 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:01.173336 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:01.173336 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:01.173336 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:01.173336 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:01.173336 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:01.173336 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:01.173336 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:01.173336 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:01.173336 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:01.174194 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:01.174194 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:01.174194 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:01.174194 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:01.174194 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 09:46:01.174194 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 09:46:01.174194 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 09:46:01.174194 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 09:46:01.174194 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 09:46:01.174194 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 09:46:01.174194 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 02 09:46:01.174194 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:46:01.174194 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 165/330] 10.222.0.64 () {56 vars in 1099 bytes} [Tue Aug 2 09:46:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 09:46:01.177701 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e61684-11b2-4d0a-935f-ceb49dd54db6 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:01.181259 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896f949a-ea85-4b3c-bf95-15210721e0e3 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:01.221695 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:01.303134 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896f949a-ea85-4b3c-bf95-15210721e0e3 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:01.307739 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e61684-11b2-4d0a-935f-ceb49dd54db6 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:01.349963 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3788db-59ff-40ce-ac84-a16fb241139c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:01.351200 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3788db-59ff-40ce-ac84-a16fb241139c None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:01.351744 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3788db-59ff-40ce-ac84-a16fb241139c None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:01.352019 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3788db-59ff-40ce-ac84-a16fb241139c None admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:01.352803 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3788db-59ff-40ce-ac84-a16fb241139c None admin] RBAC: Adding query filter params (name=member) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:01.353110 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3788db-59ff-40ce-ac84-a16fb241139c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:01.354519 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3788db-59ff-40ce-ac84-a16fb241139c None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:01.358178 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe5081b-cea6-480d-8436-75e3de3b70bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:01.359164 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe5081b-cea6-480d-8436-75e3de3b70bb None admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:01.359633 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe5081b-cea6-480d-8436-75e3de3b70bb None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:01.360090 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe5081b-cea6-480d-8436-75e3de3b70bb None admin] PATH_INFO: `/v3/system/users/cc0f20e1976e4ef4843f697366ea0730/roles/7fce0045dfa049bdb7ae0d2424de9f32` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:01.363177 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe5081b-cea6-480d-8436-75e3de3b70bb None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=cc0f20e1976e4ef4843f697366ea0730, role_id=7fce0045dfa049bdb7ae0d2424de9f32)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:01.365201 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe5081b-cea6-480d-8436-75e3de3b70bb None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:01.366015 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 166/331] 10.222.0.64 () {56 vars in 1101 bytes} [Tue Aug 2 09:46:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:46:01.372650 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3788db-59ff-40ce-ac84-a16fb241139c None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:01.378533 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 166/332] 10.222.0.64 () {58 vars in 1334 bytes} [Tue Aug 2 09:46:01 2022] PUT /identity/v3/system/users/cc0f20e1976e4ef4843f697366ea0730/roles/7fce0045dfa049bdb7ae0d2424de9f32 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:46:01.507691 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:01.520633 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3788db-59ff-40ce-ac84-a16fb241139c None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:01.567256 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24097968-c5d2-42e7-9b5e-ac2b797c0add None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:01.568299 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24097968-c5d2-42e7-9b5e-ac2b797c0add None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:01.568775 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24097968-c5d2-42e7-9b5e-ac2b797c0add None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:01.569234 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24097968-c5d2-42e7-9b5e-ac2b797c0add None admin] PATH_INFO: `/v3/users/e10a792b89354f1290e50e259936e4da` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:01.570863 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24097968-c5d2-42e7-9b5e-ac2b797c0add None admin] RBAC: Authorizing `identity:get_user(user_id=e10a792b89354f1290e50e259936e4da)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:01.572656 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24097968-c5d2-42e7-9b5e-ac2b797c0add None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:01.574658 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 167/333] 10.222.0.64 () {56 vars in 1177 bytes} [Tue Aug 2 09:46:01 2022] GET /identity/v3/users/e10a792b89354f1290e50e259936e4da => generated 293 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:46:01.581048 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe5081b-cea6-480d-8436-75e3de3b70bb None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:01.670571 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:01.730999 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe5081b-cea6-480d-8436-75e3de3b70bb None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:01.782511 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ef9b67-2c79-4f47-be4f-4e8c5fd1a3ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:01.783078 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ef9b67-2c79-4f47-be4f-4e8c5fd1a3ff None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:01.783367 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ef9b67-2c79-4f47-be4f-4e8c5fd1a3ff None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:01.783693 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ef9b67-2c79-4f47-be4f-4e8c5fd1a3ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:01.784548 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ef9b67-2c79-4f47-be4f-4e8c5fd1a3ff None admin] RBAC: Adding query filter params (role.id=66c1074780954f8e87ed11a5c4e8c36a, user.id=e10a792b89354f1290e50e259936e4da, scope.system=all) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:01.784863 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ef9b67-2c79-4f47-be4f-4e8c5fd1a3ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:01.786335 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ef9b67-2c79-4f47-be4f-4e8c5fd1a3ff None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:01.805586 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 167/334] 10.222.0.64 () {56 vars in 1308 bytes} [Tue Aug 2 09:46:01 2022] GET /identity/v3/role_assignments?user.id=e10a792b89354f1290e50e259936e4da&scope.system=all&role.id=66c1074780954f8e87ed11a5c4e8c36a => generated 510 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:46:03.644288 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94090b1-45bf-4fdc-99ad-096c8f2f55e6 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:03.644840 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94090b1-45bf-4fdc-99ad-096c8f2f55e6 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:03.645108 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94090b1-45bf-4fdc-99ad-096c8f2f55e6 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:03.646685 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 168/335] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:46:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:46:03.652724 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f1de89-d1a9-4a91-b8ef-8148e3960d82 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:03.653207 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f1de89-d1a9-4a91-b8ef-8148e3960d82 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:03.653308 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f1de89-d1a9-4a91-b8ef-8148e3960d82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:03.675313 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-b9f1de89-d1a9-4a91-b8ef-8148e3960d82 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:46:03.677011 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-b9f1de89-d1a9-4a91-b8ef-8148e3960d82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:46:03.776461 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 168/336] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:46:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 09:46:03.824008 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94090b1-45bf-4fdc-99ad-096c8f2f55e6 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:03.852692 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed67d4e-3d84-44dc-96d4-5ac559b01ed8 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:03.852980 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed67d4e-3d84-44dc-96d4-5ac559b01ed8 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:03.853289 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed67d4e-3d84-44dc-96d4-5ac559b01ed8 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:03.854595 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 169/337] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:46:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 09:46:03.866523 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbe9d73-ab3f-4c58-87ce-7acd51ab1ef2 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:03.866896 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbe9d73-ab3f-4c58-87ce-7acd51ab1ef2 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:03.867147 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbe9d73-ab3f-4c58-87ce-7acd51ab1ef2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:03.919630 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-bcbe9d73-ab3f-4c58-87ce-7acd51ab1ef2 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:46:03.920788 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-bcbe9d73-ab3f-4c58-87ce-7acd51ab1ef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:46:04.024197 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:04.047638 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94090b1-45bf-4fdc-99ad-096c8f2f55e6 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:04.078117 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 170/338] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:46:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 09:46:04.094152 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977a9a7f-5e83-4650-883a-9df76676e922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:04.094819 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977a9a7f-5e83-4650-883a-9df76676e922 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:04.095107 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977a9a7f-5e83-4650-883a-9df76676e922 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:04.095386 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977a9a7f-5e83-4650-883a-9df76676e922 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:04.116221 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbe9d73-ab3f-4c58-87ce-7acd51ab1ef2 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:04.118209 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977a9a7f-5e83-4650-883a-9df76676e922 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 09:46:04.123731 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-977a9a7f-5e83-4650-883a-9df76676e922 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 02 09:46:04.123731 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:04.123731 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:04.123731 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:04.123731 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:04.123731 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:04.123731 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:04.123731 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:04.123731 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:04.123731 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:04.123731 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:04.123731 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:04.123731 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 09:46:04.123731 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 09:46:04.123731 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 09:46:04.123731 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 09:46:04.125670 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 09:46:04.125670 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 09:46:04.125670 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 09:46:04.125670 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 09:46:04.125670 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:04.125670 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:04.125670 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:04.125670 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:04.125670 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:04.125670 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:04.125670 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:04.125670 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:04.125670 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:04.125670 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:04.125670 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:04.125670 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:04.127873 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:04.127873 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:04.127873 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:04.127873 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:04.127873 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 09:46:04.127873 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 09:46:04.127873 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 09:46:04.127873 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 09:46:04.127873 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 09:46:04.127873 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 09:46:04.127873 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 02 09:46:04.127873 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:46:04.127873 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 169/339] 10.222.0.64 () {56 vars in 1099 bytes} [Tue Aug 2 09:46:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:46:04.146735 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977a9a7f-5e83-4650-883a-9df76676e922 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:04.198974 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:04.324327 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977a9a7f-5e83-4650-883a-9df76676e922 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:04.347274 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbe9d73-ab3f-4c58-87ce-7acd51ab1ef2 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:04.373941 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70a2a68-d6f8-4146-9899-653c331ff627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:04.374896 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70a2a68-d6f8-4146-9899-653c331ff627 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:04.375412 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70a2a68-d6f8-4146-9899-653c331ff627 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:04.375830 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70a2a68-d6f8-4146-9899-653c331ff627 None admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:04.376768 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70a2a68-d6f8-4146-9899-653c331ff627 None admin] RBAC: Adding query filter params (name=member) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:04.377249 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70a2a68-d6f8-4146-9899-653c331ff627 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:04.378800 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70a2a68-d6f8-4146-9899-653c331ff627 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:04.393103 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 170/340] 10.222.0.64 () {56 vars in 1101 bytes} [Tue Aug 2 09:46:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:46:04.399324 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70a2a68-d6f8-4146-9899-653c331ff627 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:04.441951 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31c7607-285b-4d45-8bbf-b5d5495fcd2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:04.442866 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31c7607-285b-4d45-8bbf-b5d5495fcd2b None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:04.443337 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31c7607-285b-4d45-8bbf-b5d5495fcd2b None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:04.443760 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31c7607-285b-4d45-8bbf-b5d5495fcd2b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:04.471578 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31c7607-285b-4d45-8bbf-b5d5495fcd2b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 09:46:04.477212 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-e31c7607-285b-4d45-8bbf-b5d5495fcd2b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 02 09:46:04.477212 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:04.477212 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:04.477212 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:04.477212 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:04.477212 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:04.477212 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:04.477212 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:04.477212 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:04.477212 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:04.477212 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:04.477212 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:04.477212 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 09:46:04.477212 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 09:46:04.477212 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 09:46:04.477212 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 09:46:04.478293 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 09:46:04.478293 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 09:46:04.478293 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 09:46:04.478293 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 09:46:04.478293 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:04.478293 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:04.478293 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:04.478293 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:04.478293 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:04.478293 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:04.478293 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:04.478293 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:04.478293 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:04.478293 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:04.478293 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:04.478293 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:04.479273 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:04.479273 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:04.479273 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:04.479273 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:04.479273 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 09:46:04.479273 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 09:46:04.479273 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 09:46:04.479273 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 09:46:04.479273 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 09:46:04.479273 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 09:46:04.479273 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 02 09:46:04.479273 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:46:04.485679 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 171/341] 10.222.0.64 () {56 vars in 1099 bytes} [Tue Aug 2 09:46:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 370 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:46:04.487740 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404fb5f9-c9b4-408e-94ae-1fe10e98ef62 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:04.488186 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404fb5f9-c9b4-408e-94ae-1fe10e98ef62 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:04.488629 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404fb5f9-c9b4-408e-94ae-1fe10e98ef62 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:04.490132 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 172/342] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:46:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:46:04.495625 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5396e2-00e9-444e-9981-c3e08b0dcd55 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:04.496110 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5396e2-00e9-444e-9981-c3e08b0dcd55 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:04.496584 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5396e2-00e9-444e-9981-c3e08b0dcd55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:04.514349 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:04.526129 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-5c5396e2-00e9-444e-9981-c3e08b0dcd55 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:46:04.527747 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-5c5396e2-00e9-444e-9981-c3e08b0dcd55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:46:04.533756 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70a2a68-d6f8-4146-9899-653c331ff627 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:04.573531 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6e541c-9b95-454a-a789-43cebfe2c21e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:04.574451 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6e541c-9b95-454a-a789-43cebfe2c21e None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:04.574879 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6e541c-9b95-454a-a789-43cebfe2c21e None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:04.575426 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6e541c-9b95-454a-a789-43cebfe2c21e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:04.616931 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:04.652220 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6e541c-9b95-454a-a789-43cebfe2c21e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:04.654289 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6e541c-9b95-454a-a789-43cebfe2c21e None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:04.734575 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 173/343] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:46:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 09:46:04.735393 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5396e2-00e9-444e-9981-c3e08b0dcd55 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:04.754216 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:04.777621 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 171/344] 10.222.0.64 () {58 vars in 1282 bytes} [Tue Aug 2 09:46:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 02 09:46:04.783279 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6e541c-9b95-454a-a789-43cebfe2c21e None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:04.913271 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6e541c-9b95-454a-a789-43cebfe2c21e None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:04.924414 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5396e2-00e9-444e-9981-c3e08b0dcd55 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:04.978304 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0ff6a6-8375-458e-8541-adebbe2336b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:04.978956 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0ff6a6-8375-458e-8541-adebbe2336b2 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:04.979244 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0ff6a6-8375-458e-8541-adebbe2336b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:04.979525 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0ff6a6-8375-458e-8541-adebbe2336b2 None admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:04.980102 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3683769-ac9a-4590-8275-fc15b2ff889e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:04.980297 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0ff6a6-8375-458e-8541-adebbe2336b2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:04.980621 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0ff6a6-8375-458e-8541-adebbe2336b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:04.981237 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3683769-ac9a-4590-8275-fc15b2ff889e None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:04.981658 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3683769-ac9a-4590-8275-fc15b2ff889e None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:04.982028 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0ff6a6-8375-458e-8541-adebbe2336b2 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:04.982239 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3683769-ac9a-4590-8275-fc15b2ff889e None admin] PATH_INFO: `/v3/projects/ae78e3e21c0448c0b412e161cc374652` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:04.983591 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3683769-ac9a-4590-8275-fc15b2ff889e None admin] RBAC: Authorizing `identity:get_project(project_id=ae78e3e21c0448c0b412e161cc374652)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:04.985123 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3683769-ac9a-4590-8275-fc15b2ff889e None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:04.987310 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 172/345] 10.222.0.64 () {56 vars in 1186 bytes} [Tue Aug 2 09:46:04 2022] GET /identity/v3/projects/ae78e3e21c0448c0b412e161cc374652 => generated 298 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:46:04.988481 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3683769-ac9a-4590-8275-fc15b2ff889e None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:04.993831 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 174/346] 10.222.0.64 () {56 vars in 1101 bytes} [Tue Aug 2 09:46:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:04.995402 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc7adec-570c-43e4-89fc-f5bf2a35b556 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:04.995915 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc7adec-570c-43e4-89fc-f5bf2a35b556 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:04.996423 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc7adec-570c-43e4-89fc-f5bf2a35b556 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:04.998264 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 175/347] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:46:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:46:04.999182 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc7adec-570c-43e4-89fc-f5bf2a35b556 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:05.045430 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:05.078005 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:05.148047 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3683769-ac9a-4590-8275-fc15b2ff889e None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:05.151299 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc7adec-570c-43e4-89fc-f5bf2a35b556 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:05.203201 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443e7bc0-7cf4-44cb-aa36-58a714d9cbc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:05.204460 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443e7bc0-7cf4-44cb-aa36-58a714d9cbc8 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:05.204915 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443e7bc0-7cf4-44cb-aa36-58a714d9cbc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:05.205910 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443e7bc0-7cf4-44cb-aa36-58a714d9cbc8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:05.213169 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1954e0a6-1e6d-48b1-abc1-c4e28630e61d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:05.214211 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1954e0a6-1e6d-48b1-abc1-c4e28630e61d None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:05.214667 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1954e0a6-1e6d-48b1-abc1-c4e28630e61d None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:05.215162 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1954e0a6-1e6d-48b1-abc1-c4e28630e61d None admin] PATH_INFO: `/v3/groups/da5d1ad048174277ae2e03e61988bad9` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:05.221309 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443e7bc0-7cf4-44cb-aa36-58a714d9cbc8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 09:46:05.228754 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1954e0a6-1e6d-48b1-abc1-c4e28630e61d None admin] RBAC: Authorizing `identity:get_group(group_id=da5d1ad048174277ae2e03e61988bad9)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:05.228754 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1954e0a6-1e6d-48b1-abc1-c4e28630e61d None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:05.229048 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-443e7bc0-7cf4-44cb-aa36-58a714d9cbc8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 09:46:05.229048 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:05.229048 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:05.229048 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:05.229048 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:05.229048 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:05.229048 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:05.229048 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:05.229048 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:05.229048 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:05.229048 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:05.229048 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:05.229048 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 09:46:05.229048 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 09:46:05.229048 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 09:46:05.229048 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 09:46:05.229981 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 09:46:05.229981 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 09:46:05.229981 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 09:46:05.229981 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 09:46:05.229981 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:05.229981 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:05.229981 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:05.229981 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:05.229981 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:05.229981 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:05.229981 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:05.229981 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:05.229981 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:05.229981 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:05.229981 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:05.229981 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:05.230804 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:05.230804 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:05.230804 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:05.230804 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:05.230804 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 09:46:05.230804 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 09:46:05.230804 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 09:46:05.230804 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 09:46:05.230804 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 09:46:05.230804 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 09:46:05.230804 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 09:46:05.230804 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:46:05.230804 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 173/348] 10.222.0.64 () {56 vars in 1114 bytes} [Tue Aug 2 09:46:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:46:05.231477 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 176/349] 10.222.0.64 () {56 vars in 1180 bytes} [Tue Aug 2 09:46:04 2022] GET /identity/v3/groups/da5d1ad048174277ae2e03e61988bad9 => generated 226 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:05.231477 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1954e0a6-1e6d-48b1-abc1-c4e28630e61d None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:05.239446 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2251c45b-6f8c-4d8d-8e34-d522f301f8ea None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:05.239813 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2251c45b-6f8c-4d8d-8e34-d522f301f8ea None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:05.240235 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2251c45b-6f8c-4d8d-8e34-d522f301f8ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:05.268683 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-2251c45b-6f8c-4d8d-8e34-d522f301f8ea None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:46:05.269997 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-2251c45b-6f8c-4d8d-8e34-d522f301f8ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:46:05.376930 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:05.383408 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 174/350] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:46:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 09:46:05.384908 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f0aaf5-295a-4dc4-9844-f2d6c95b2b2f None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:05.385316 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f0aaf5-295a-4dc4-9844-f2d6c95b2b2f None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:05.385595 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f0aaf5-295a-4dc4-9844-f2d6c95b2b2f None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:05.385761 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1954e0a6-1e6d-48b1-abc1-c4e28630e61d None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:05.387244 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 175/351] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:46:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:46:05.388056 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f0aaf5-295a-4dc4-9844-f2d6c95b2b2f None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:05.445784 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde9cf56-de95-47ac-9042-f872109531ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:05.446551 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde9cf56-de95-47ac-9042-f872109531ea None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:05.446837 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde9cf56-de95-47ac-9042-f872109531ea None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:05.447111 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde9cf56-de95-47ac-9042-f872109531ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:05.452526 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:05.539993 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f0aaf5-295a-4dc4-9844-f2d6c95b2b2f None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:05.543502 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde9cf56-de95-47ac-9042-f872109531ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:05.545901 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde9cf56-de95-47ac-9042-f872109531ea None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:05.596667 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3b7bdb-e372-44b4-8aa7-e5458a5f148e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:05.597315 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3b7bdb-e372-44b4-8aa7-e5458a5f148e None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:05.597602 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3b7bdb-e372-44b4-8aa7-e5458a5f148e None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:05.597876 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3b7bdb-e372-44b4-8aa7-e5458a5f148e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:05.598703 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3b7bdb-e372-44b4-8aa7-e5458a5f148e None admin] RBAC: Adding query filter params (group.id=da5d1ad048174277ae2e03e61988bad9, role.id=66c1074780954f8e87ed11a5c4e8c36a, scope.project.id=ae78e3e21c0448c0b412e161cc374652) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:05.599015 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3b7bdb-e372-44b4-8aa7-e5458a5f148e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:05.602379 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3b7bdb-e372-44b4-8aa7-e5458a5f148e None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:05.614589 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 176/352] 10.222.0.64 () {56 vars in 1376 bytes} [Tue Aug 2 09:46:05 2022] GET /identity/v3/role_assignments?group.id=da5d1ad048174277ae2e03e61988bad9&scope.project.id=ae78e3e21c0448c0b412e161cc374652&role.id=66c1074780954f8e87ed11a5c4e8c36a => generated 260 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:05.615630 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3b7bdb-e372-44b4-8aa7-e5458a5f148e None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:05.702274 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:05.719224 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 177/353] 10.222.0.64 () {58 vars in 1282 bytes} [Tue Aug 2 09:46:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 09:46:05.721950 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02013c6d-0d05-4a53-807f-696c5618ca5a None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:05.722826 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02013c6d-0d05-4a53-807f-696c5618ca5a None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:05.723552 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02013c6d-0d05-4a53-807f-696c5618ca5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:05.753241 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-02013c6d-0d05-4a53-807f-696c5618ca5a None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:46:05.755205 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-02013c6d-0d05-4a53-807f-696c5618ca5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:46:05.777597 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3b7bdb-e372-44b4-8aa7-e5458a5f148e None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:05.787019 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:05.838338 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abed5868-141b-4225-8599-5ce73da40145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:05.839152 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abed5868-141b-4225-8599-5ce73da40145 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:05.839664 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abed5868-141b-4225-8599-5ce73da40145 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:05.839990 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abed5868-141b-4225-8599-5ce73da40145 None admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:05.843691 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abed5868-141b-4225-8599-5ce73da40145 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:05.843691 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abed5868-141b-4225-8599-5ce73da40145 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:05.843691 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abed5868-141b-4225-8599-5ce73da40145 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:05.854685 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 177/354] 10.222.0.64 () {56 vars in 1111 bytes} [Tue Aug 2 09:46:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:05.855575 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abed5868-141b-4225-8599-5ce73da40145 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:05.888961 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 178/355] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:46:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 09:46:05.891025 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02013c6d-0d05-4a53-807f-696c5618ca5a None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:06.001303 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abed5868-141b-4225-8599-5ce73da40145 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:06.040585 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02013c6d-0d05-4a53-807f-696c5618ca5a None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:06.053775 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5c2b99-67ab-493f-b462-d18f9a995d33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:06.054620 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5c2b99-67ab-493f-b462-d18f9a995d33 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:06.054915 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5c2b99-67ab-493f-b462-d18f9a995d33 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:06.055194 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5c2b99-67ab-493f-b462-d18f9a995d33 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:06.070290 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:06.075295 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5c2b99-67ab-493f-b462-d18f9a995d33 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 09:46:06.083769 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-7f5c2b99-67ab-493f-b462-d18f9a995d33 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 02 09:46:06.083769 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:06.083769 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:06.083769 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:06.083769 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:06.083769 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:06.083769 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:06.083769 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:06.083769 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:06.083769 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:06.083769 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:06.083769 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:06.083769 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 09:46:06.083769 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 09:46:06.083769 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 09:46:06.083769 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 09:46:06.086188 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 09:46:06.086188 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 09:46:06.086188 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 09:46:06.086188 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 09:46:06.086188 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:06.086188 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:06.086188 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:06.086188 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:06.086188 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:06.086188 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:06.086188 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:06.086188 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:06.086188 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:06.086188 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:06.086188 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:06.086188 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:06.088508 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:06.088508 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:06.088508 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:06.088508 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:06.088508 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 09:46:06.088508 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 09:46:06.088508 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 09:46:06.088508 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 09:46:06.088508 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 09:46:06.088508 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 09:46:06.088508 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 02 09:46:06.088508 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:46:06.089313 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3eb1db9-4b02-4dfb-8ea7-57f59bff4e05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:06.089313 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eb1db9-4b02-4dfb-8ea7-57f59bff4e05 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:06.089313 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eb1db9-4b02-4dfb-8ea7-57f59bff4e05 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:06.089313 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eb1db9-4b02-4dfb-8ea7-57f59bff4e05 None admin] PATH_INFO: `/v3/projects/bc7fa79dde7e42ae92a2a42495efdcc2` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:06.089593 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 178/356] 10.222.0.64 () {56 vars in 1099 bytes} [Tue Aug 2 09:46:05 2022] GET /identity/v3/roles/reader => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:46:06.089593 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5c2b99-67ab-493f-b462-d18f9a995d33 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:06.090100 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3eb1db9-4b02-4dfb-8ea7-57f59bff4e05 None admin] RBAC: Authorizing `identity:get_project(project_id=bc7fa79dde7e42ae92a2a42495efdcc2)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:06.090100 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3eb1db9-4b02-4dfb-8ea7-57f59bff4e05 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:06.094609 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 179/357] 10.222.0.64 () {56 vars in 1186 bytes} [Tue Aug 2 09:46:05 2022] GET /identity/v3/projects/bc7fa79dde7e42ae92a2a42495efdcc2 => generated 302 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:06.094609 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6215c307-e5e3-4da7-8b45-41833da0b45d None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:06.094609 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6215c307-e5e3-4da7-8b45-41833da0b45d None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:06.094609 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6215c307-e5e3-4da7-8b45-41833da0b45d None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:06.094609 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 180/358] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:46:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:46:06.096662 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6215c307-e5e3-4da7-8b45-41833da0b45d None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:06.137050 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:06.243755 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5c2b99-67ab-493f-b462-d18f9a995d33 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:06.253394 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6215c307-e5e3-4da7-8b45-41833da0b45d None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:06.290832 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecb100a-c02d-4f9d-a715-3489e187c32d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:06.291486 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecb100a-c02d-4f9d-a715-3489e187c32d None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:06.291773 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecb100a-c02d-4f9d-a715-3489e187c32d None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:06.292076 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecb100a-c02d-4f9d-a715-3489e187c32d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:06.308639 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c58169-c024-4efc-b55d-9c6ccdb1c402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:06.309987 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c58169-c024-4efc-b55d-9c6ccdb1c402 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:06.310649 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c58169-c024-4efc-b55d-9c6ccdb1c402 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:06.311339 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c58169-c024-4efc-b55d-9c6ccdb1c402 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:06.326461 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c58169-c024-4efc-b55d-9c6ccdb1c402 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 09:46:06.333573 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-c2c58169-c024-4efc-b55d-9c6ccdb1c402 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 09:46:06.333573 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:06.333573 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:06.333573 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:06.333573 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:06.333573 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:06.333573 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:06.333573 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:06.333573 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:06.333573 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:06.333573 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:06.333573 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:06.333573 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 09:46:06.333573 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 09:46:06.333573 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 09:46:06.333573 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 09:46:06.334541 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 09:46:06.334541 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 09:46:06.334541 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 09:46:06.334541 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 09:46:06.334541 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:06.334541 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:06.334541 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:06.334541 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:06.334541 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:06.334541 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:06.334541 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:06.334541 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:06.334541 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:06.334541 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:06.334541 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:06.334541 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:06.335461 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:06.335461 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:06.335461 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:06.335461 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:06.335461 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 09:46:06.335461 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 09:46:06.335461 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 09:46:06.335461 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 09:46:06.335461 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 09:46:06.335461 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 09:46:06.335461 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 09:46:06.335461 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:46:06.335461 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 181/359] 10.222.0.64 () {56 vars in 1114 bytes} [Tue Aug 2 09:46:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:46:06.336139 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c58169-c024-4efc-b55d-9c6ccdb1c402 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:06.408435 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:06.409356 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecb100a-c02d-4f9d-a715-3489e187c32d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:06.414136 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecb100a-c02d-4f9d-a715-3489e187c32d None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:06.449673 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:06.486138 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c58169-c024-4efc-b55d-9c6ccdb1c402 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:06.536931 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119fca2a-90be-4b66-a072-4b955a773a1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:06.537953 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119fca2a-90be-4b66-a072-4b955a773a1a None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:06.538402 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119fca2a-90be-4b66-a072-4b955a773a1a None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:06.538679 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119fca2a-90be-4b66-a072-4b955a773a1a None admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:06.539418 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119fca2a-90be-4b66-a072-4b955a773a1a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:06.539730 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119fca2a-90be-4b66-a072-4b955a773a1a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:06.541166 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119fca2a-90be-4b66-a072-4b955a773a1a None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:06.559014 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 179/360] 10.222.0.64 () {58 vars in 1282 bytes} [Tue Aug 2 09:46:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 09:46:06.559014 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecb100a-c02d-4f9d-a715-3489e187c32d None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:06.562075 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 182/361] 10.222.0.64 () {56 vars in 1101 bytes} [Tue Aug 2 09:46:06 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:06.563599 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a72815-dc4b-48c8-b3ec-b190f94755c8 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:06.564025 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a72815-dc4b-48c8-b3ec-b190f94755c8 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:06.564471 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a72815-dc4b-48c8-b3ec-b190f94755c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:06.592897 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-07a72815-dc4b-48c8-b3ec-b190f94755c8 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:46:06.595202 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-07a72815-dc4b-48c8-b3ec-b190f94755c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:46:06.682317 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecb100a-c02d-4f9d-a715-3489e187c32d None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:06.688304 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 183/362] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:46:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 09:46:06.689167 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a72815-dc4b-48c8-b3ec-b190f94755c8 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:06.713088 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:06.722608 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078f8350-1128-47b8-9720-82197c7c74ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:06.723338 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078f8350-1128-47b8-9720-82197c7c74ff None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:06.723973 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078f8350-1128-47b8-9720-82197c7c74ff None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:06.724362 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078f8350-1128-47b8-9720-82197c7c74ff None admin] PATH_INFO: `/v3/groups/da5d1ad048174277ae2e03e61988bad9` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:06.726439 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078f8350-1128-47b8-9720-82197c7c74ff None admin] RBAC: Authorizing `identity:get_group(group_id=da5d1ad048174277ae2e03e61988bad9)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:06.729478 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078f8350-1128-47b8-9720-82197c7c74ff None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:06.732148 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 180/363] 10.222.0.64 () {56 vars in 1180 bytes} [Tue Aug 2 09:46:06 2022] GET /identity/v3/groups/da5d1ad048174277ae2e03e61988bad9 => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:06.733059 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078f8350-1128-47b8-9720-82197c7c74ff None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:06.742940 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:06.829237 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a72815-dc4b-48c8-b3ec-b190f94755c8 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:06.888121 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078f8350-1128-47b8-9720-82197c7c74ff None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:06.910579 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c86185-e1e0-4088-8237-61225948bb64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:06.911233 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c86185-e1e0-4088-8237-61225948bb64 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:06.911702 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c86185-e1e0-4088-8237-61225948bb64 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:06.911988 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c86185-e1e0-4088-8237-61225948bb64 None admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:06.912775 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c86185-e1e0-4088-8237-61225948bb64 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:06.913088 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c86185-e1e0-4088-8237-61225948bb64 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:06.914899 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c86185-e1e0-4088-8237-61225948bb64 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:06.930029 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 184/364] 10.222.0.64 () {56 vars in 1111 bytes} [Tue Aug 2 09:46:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:06.931276 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c86185-e1e0-4088-8237-61225948bb64 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:06.948840 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb4b96d-2d9b-4cf2-82e6-c3c637d585f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:06.949908 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb4b96d-2d9b-4cf2-82e6-c3c637d585f5 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:06.950205 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb4b96d-2d9b-4cf2-82e6-c3c637d585f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:06.950487 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb4b96d-2d9b-4cf2-82e6-c3c637d585f5 None admin] PATH_INFO: `/v3/projects/ae78e3e21c0448c0b412e161cc374652` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:06.952512 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb4b96d-2d9b-4cf2-82e6-c3c637d585f5 None admin] RBAC: Authorizing `identity:get_project(project_id=ae78e3e21c0448c0b412e161cc374652)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:06.954291 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb4b96d-2d9b-4cf2-82e6-c3c637d585f5 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:06.956960 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 181/365] 10.222.0.64 () {56 vars in 1186 bytes} [Tue Aug 2 09:46:06 2022] GET /identity/v3/projects/ae78e3e21c0448c0b412e161cc374652 => generated 298 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:06.959079 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb4b96d-2d9b-4cf2-82e6-c3c637d585f5 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:07.053979 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:07.077560 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c86185-e1e0-4088-8237-61225948bb64 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:07.087017 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:07.113682 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb4b96d-2d9b-4cf2-82e6-c3c637d585f5 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:07.135382 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48cd465-c1d4-438c-9728-22ca1cae633e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:07.136028 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48cd465-c1d4-438c-9728-22ca1cae633e None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:07.136342 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48cd465-c1d4-438c-9728-22ca1cae633e None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:07.136617 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48cd465-c1d4-438c-9728-22ca1cae633e None admin] PATH_INFO: `/v3/users/cc0f20e1976e4ef4843f697366ea0730` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:07.137860 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48cd465-c1d4-438c-9728-22ca1cae633e None admin] RBAC: Authorizing `identity:get_user(user_id=cc0f20e1976e4ef4843f697366ea0730)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:07.139306 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48cd465-c1d4-438c-9728-22ca1cae633e None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:07.141208 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 185/366] 10.222.0.64 () {56 vars in 1177 bytes} [Tue Aug 2 09:46:06 2022] GET /identity/v3/users/cc0f20e1976e4ef4843f697366ea0730 => generated 293 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:07.141888 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48cd465-c1d4-438c-9728-22ca1cae633e None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:07.170328 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376ec4ae-04b2-4ee4-b060-354c21314794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:07.170974 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376ec4ae-04b2-4ee4-b060-354c21314794 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:07.171254 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376ec4ae-04b2-4ee4-b060-354c21314794 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:07.171525 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376ec4ae-04b2-4ee4-b060-354c21314794 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:07.188110 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376ec4ae-04b2-4ee4-b060-354c21314794 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 09:46:07.196453 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-376ec4ae-04b2-4ee4-b060-354c21314794 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 09:46:07.196453 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:07.196453 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:07.196453 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:07.196453 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:07.196453 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:07.196453 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:07.196453 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:07.196453 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:07.196453 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:07.196453 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:07.196453 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:07.196453 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 09:46:07.196453 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 09:46:07.196453 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 09:46:07.196453 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 09:46:07.197462 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 09:46:07.197462 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 09:46:07.197462 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 09:46:07.197462 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 09:46:07.197462 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:07.197462 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:07.197462 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:07.197462 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:07.197462 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:07.197462 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:07.197462 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:07.197462 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:07.197462 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:07.197462 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:07.197462 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:07.197462 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:07.198340 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:07.198340 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:07.198340 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:07.198340 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:07.198340 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 09:46:07.198340 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 09:46:07.198340 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 09:46:07.198340 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 09:46:07.198340 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 09:46:07.198340 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 09:46:07.198340 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 09:46:07.198340 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:46:07.198340 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 182/367] 10.222.0.64 () {56 vars in 1096 bytes} [Tue Aug 2 09:46:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:46:07.198340 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376ec4ae-04b2-4ee4-b060-354c21314794 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:07.305576 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48cd465-c1d4-438c-9728-22ca1cae633e None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:07.340763 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376ec4ae-04b2-4ee4-b060-354c21314794 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:07.351661 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371aa7f3-f5b1-48a2-9654-76c00f998bea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:07.352327 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371aa7f3-f5b1-48a2-9654-76c00f998bea None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:07.352653 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371aa7f3-f5b1-48a2-9654-76c00f998bea None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:07.352895 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371aa7f3-f5b1-48a2-9654-76c00f998bea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:07.353737 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371aa7f3-f5b1-48a2-9654-76c00f998bea None admin] RBAC: Adding query filter params (group.id=da5d1ad048174277ae2e03e61988bad9, role.id=66c1074780954f8e87ed11a5c4e8c36a, scope.project.id=bc7fa79dde7e42ae92a2a42495efdcc2) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:07.354531 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371aa7f3-f5b1-48a2-9654-76c00f998bea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:07.355503 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371aa7f3-f5b1-48a2-9654-76c00f998bea None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:07.363207 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 186/368] 10.222.0.64 () {56 vars in 1376 bytes} [Tue Aug 2 09:46:07 2022] GET /identity/v3/role_assignments?group.id=da5d1ad048174277ae2e03e61988bad9&scope.project.id=bc7fa79dde7e42ae92a2a42495efdcc2&role.id=66c1074780954f8e87ed11a5c4e8c36a => generated 260 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:07.364054 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371aa7f3-f5b1-48a2-9654-76c00f998bea None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:07.378016 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:07.391962 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2086264-12cc-41f6-8054-a9dc1a8ef644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:07.392790 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2086264-12cc-41f6-8054-a9dc1a8ef644 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:07.393197 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2086264-12cc-41f6-8054-a9dc1a8ef644 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:07.393583 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2086264-12cc-41f6-8054-a9dc1a8ef644 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:07.429061 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:07.501422 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2086264-12cc-41f6-8054-a9dc1a8ef644 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:07.504304 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2086264-12cc-41f6-8054-a9dc1a8ef644 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:07.514419 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371aa7f3-f5b1-48a2-9654-76c00f998bea None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:07.575474 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625b91a9-e4c8-44b4-8910-01bd3834a4f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:07.576454 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625b91a9-e4c8-44b4-8910-01bd3834a4f0 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:07.576951 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625b91a9-e4c8-44b4-8910-01bd3834a4f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:07.577385 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625b91a9-e4c8-44b4-8910-01bd3834a4f0 None admin] PATH_INFO: `/v3/groups/da5d1ad048174277ae2e03e61988bad9` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:07.578778 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625b91a9-e4c8-44b4-8910-01bd3834a4f0 None admin] RBAC: Authorizing `identity:get_group(group_id=da5d1ad048174277ae2e03e61988bad9)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:07.582218 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625b91a9-e4c8-44b4-8910-01bd3834a4f0 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:07.584760 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 187/369] 10.222.0.64 () {56 vars in 1180 bytes} [Tue Aug 2 09:46:07 2022] GET /identity/v3/groups/da5d1ad048174277ae2e03e61988bad9 => generated 226 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:07.585867 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625b91a9-e4c8-44b4-8910-01bd3834a4f0 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:07.677051 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 183/370] 10.222.0.64 () {58 vars in 1282 bytes} [Tue Aug 2 09:46:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 09:46:07.678414 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2086264-12cc-41f6-8054-a9dc1a8ef644 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:07.707796 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625b91a9-e4c8-44b4-8910-01bd3834a4f0 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:07.715740 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:07.719526 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:07.755206 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdcc076-4a8a-4977-931c-80bc298fcac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:07.755835 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdcc076-4a8a-4977-931c-80bc298fcac4 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:07.756122 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdcc076-4a8a-4977-931c-80bc298fcac4 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:07.756459 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdcc076-4a8a-4977-931c-80bc298fcac4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:07.757272 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdcc076-4a8a-4977-931c-80bc298fcac4 None admin] RBAC: Adding query filter params (role.id=7fce0045dfa049bdb7ae0d2424de9f32, user.id=cc0f20e1976e4ef4843f697366ea0730, scope.system=all) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:07.757587 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdcc076-4a8a-4977-931c-80bc298fcac4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:07.759055 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdcc076-4a8a-4977-931c-80bc298fcac4 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:07.774669 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 188/371] 10.222.0.64 () {56 vars in 1308 bytes} [Tue Aug 2 09:46:07 2022] GET /identity/v3/role_assignments?user.id=cc0f20e1976e4ef4843f697366ea0730&scope.system=all&role.id=7fce0045dfa049bdb7ae0d2424de9f32 => generated 510 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:07.775549 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdcc076-4a8a-4977-931c-80bc298fcac4 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:07.836012 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2086264-12cc-41f6-8054-a9dc1a8ef644 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:07.906824 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97891b0d-51a5-4995-a180-9ecdc7e588d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:07.907487 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97891b0d-51a5-4995-a180-9ecdc7e588d1 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:07.907725 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97891b0d-51a5-4995-a180-9ecdc7e588d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:07.910175 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97891b0d-51a5-4995-a180-9ecdc7e588d1 None admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:07.910942 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97891b0d-51a5-4995-a180-9ecdc7e588d1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:07.911248 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97891b0d-51a5-4995-a180-9ecdc7e588d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:07.912807 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97891b0d-51a5-4995-a180-9ecdc7e588d1 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:07.922863 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 184/372] 10.222.0.64 () {56 vars in 1099 bytes} [Tue Aug 2 09:46:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:07.923734 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97891b0d-51a5-4995-a180-9ecdc7e588d1 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:07.958780 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdcc076-4a8a-4977-931c-80bc298fcac4 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:08.017695 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c640a1f-21a0-4f3b-a8b5-22e3d61406d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:08.018842 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c640a1f-21a0-4f3b-a8b5-22e3d61406d7 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:08.019128 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c640a1f-21a0-4f3b-a8b5-22e3d61406d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:08.019407 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c640a1f-21a0-4f3b-a8b5-22e3d61406d7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:08.021026 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c640a1f-21a0-4f3b-a8b5-22e3d61406d7 None admin] RBAC: Adding query filter params (group.id=da5d1ad048174277ae2e03e61988bad9, role.id=06dfd237063c42a4bff1ad8ca29c07b0, scope.project.id=ae78e3e21c0448c0b412e161cc374652) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:08.021351 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c640a1f-21a0-4f3b-a8b5-22e3d61406d7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:08.022955 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c640a1f-21a0-4f3b-a8b5-22e3d61406d7 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:08.031989 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 189/373] 10.222.0.64 () {56 vars in 1376 bytes} [Tue Aug 2 09:46:07 2022] GET /identity/v3/role_assignments?group.id=da5d1ad048174277ae2e03e61988bad9&scope.project.id=ae78e3e21c0448c0b412e161cc374652&role.id=06dfd237063c42a4bff1ad8ca29c07b0 => generated 260 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:08.033557 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c640a1f-21a0-4f3b-a8b5-22e3d61406d7 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:08.092291 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:08.101653 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:08.118086 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97891b0d-51a5-4995-a180-9ecdc7e588d1 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:08.166703 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44a025c-2b30-4b25-a2fd-aad1b681048f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:08.167437 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44a025c-2b30-4b25-a2fd-aad1b681048f None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:08.167723 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44a025c-2b30-4b25-a2fd-aad1b681048f None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:08.168002 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44a025c-2b30-4b25-a2fd-aad1b681048f None admin] PATH_INFO: `/v3/projects/bc7fa79dde7e42ae92a2a42495efdcc2` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:08.169330 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44a025c-2b30-4b25-a2fd-aad1b681048f None admin] RBAC: Authorizing `identity:get_project(project_id=bc7fa79dde7e42ae92a2a42495efdcc2)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:08.170839 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44a025c-2b30-4b25-a2fd-aad1b681048f None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:08.172949 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 185/374] 10.222.0.64 () {56 vars in 1186 bytes} [Tue Aug 2 09:46:07 2022] GET /identity/v3/projects/bc7fa79dde7e42ae92a2a42495efdcc2 => generated 302 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:08.174138 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c42a20-49b9-43d6-b630-92bf824d5629 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:08.174427 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c42a20-49b9-43d6-b630-92bf824d5629 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:08.174703 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c42a20-49b9-43d6-b630-92bf824d5629 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:08.176320 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 186/375] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:46:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:46:08.183447 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c42a20-49b9-43d6-b630-92bf824d5629 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:08.223204 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c640a1f-21a0-4f3b-a8b5-22e3d61406d7 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:08.273325 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba8ad12-e775-441b-8ac1-7660aa279502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:08.273968 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba8ad12-e775-441b-8ac1-7660aa279502 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:08.274253 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba8ad12-e775-441b-8ac1-7660aa279502 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:08.274531 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba8ad12-e775-441b-8ac1-7660aa279502 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:08.376780 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c42a20-49b9-43d6-b630-92bf824d5629 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:08.400453 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba8ad12-e775-441b-8ac1-7660aa279502 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:08.402246 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba8ad12-e775-441b-8ac1-7660aa279502 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:08.443017 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4422ad49-ca25-44f6-8a8e-80e2d4652180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:08.443684 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4422ad49-ca25-44f6-8a8e-80e2d4652180 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:08.444616 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4422ad49-ca25-44f6-8a8e-80e2d4652180 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:08.444752 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:08.445318 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4422ad49-ca25-44f6-8a8e-80e2d4652180 None admin] PATH_INFO: `/v3/groups/da5d1ad048174277ae2e03e61988bad9` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:08.447583 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4422ad49-ca25-44f6-8a8e-80e2d4652180 None admin] RBAC: Authorizing `identity:get_group(group_id=da5d1ad048174277ae2e03e61988bad9)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:08.449082 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4422ad49-ca25-44f6-8a8e-80e2d4652180 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:08.450844 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 187/376] 10.222.0.64 () {56 vars in 1180 bytes} [Tue Aug 2 09:46:08 2022] GET /identity/v3/groups/da5d1ad048174277ae2e03e61988bad9 => generated 226 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:08.454767 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57ba222-53a7-4d1c-b050-5443b6c2cc00 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:08.455601 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57ba222-53a7-4d1c-b050-5443b6c2cc00 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:08.455920 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57ba222-53a7-4d1c-b050-5443b6c2cc00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:08.478245 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-b57ba222-53a7-4d1c-b050-5443b6c2cc00 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:46:08.479507 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-b57ba222-53a7-4d1c-b050-5443b6c2cc00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:46:08.503283 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:08.554864 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 190/377] 10.222.0.64 () {58 vars in 1282 bytes} [Tue Aug 2 09:46:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 522 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 09:46:08.556560 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba8ad12-e775-441b-8ac1-7660aa279502 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:08.582030 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 188/378] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:46:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 09:46:08.583620 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57ba222-53a7-4d1c-b050-5443b6c2cc00 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:08.702027 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba8ad12-e775-441b-8ac1-7660aa279502 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:08.738535 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:08.753997 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57ba222-53a7-4d1c-b050-5443b6c2cc00 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:08.755868 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc71fd43-4d50-45b6-964c-5ce9bf1ebe13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:08.756824 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc71fd43-4d50-45b6-964c-5ce9bf1ebe13 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:08.757251 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc71fd43-4d50-45b6-964c-5ce9bf1ebe13 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:08.757596 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc71fd43-4d50-45b6-964c-5ce9bf1ebe13 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:08.758717 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc71fd43-4d50-45b6-964c-5ce9bf1ebe13 None admin] RBAC: Adding query filter params (group.id=da5d1ad048174277ae2e03e61988bad9, role.id=06dfd237063c42a4bff1ad8ca29c07b0, scope.project.id=bc7fa79dde7e42ae92a2a42495efdcc2) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:08.758839 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc71fd43-4d50-45b6-964c-5ce9bf1ebe13 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:08.760488 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc71fd43-4d50-45b6-964c-5ce9bf1ebe13 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:08.767412 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 191/379] 10.222.0.64 () {56 vars in 1376 bytes} [Tue Aug 2 09:46:08 2022] GET /identity/v3/role_assignments?group.id=da5d1ad048174277ae2e03e61988bad9&scope.project.id=bc7fa79dde7e42ae92a2a42495efdcc2&role.id=06dfd237063c42a4bff1ad8ca29c07b0 => generated 260 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:08.769388 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc71fd43-4d50-45b6-964c-5ce9bf1ebe13 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:08.873084 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4eb3fc-fa2c-4e58-b18e-79cf86db8d34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:08.873775 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4eb3fc-fa2c-4e58-b18e-79cf86db8d34 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:08.874067 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4eb3fc-fa2c-4e58-b18e-79cf86db8d34 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:08.874348 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4eb3fc-fa2c-4e58-b18e-79cf86db8d34 None admin] PATH_INFO: `/v3/projects/34e3e6719e2a412f926ff9cdc426bf39` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:08.875723 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4eb3fc-fa2c-4e58-b18e-79cf86db8d34 None admin] RBAC: Authorizing `identity:get_project(project_id=34e3e6719e2a412f926ff9cdc426bf39)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:08.877315 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4eb3fc-fa2c-4e58-b18e-79cf86db8d34 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:08.879111 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 189/380] 10.222.0.64 () {56 vars in 1186 bytes} [Tue Aug 2 09:46:08 2022] GET /identity/v3/projects/34e3e6719e2a412f926ff9cdc426bf39 => generated 344 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:08.887366 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4eb3fc-fa2c-4e58-b18e-79cf86db8d34 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:08.900514 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc71fd43-4d50-45b6-964c-5ce9bf1ebe13 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:08.953200 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:08.994619 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9460f20a-6507-49c9-8ea6-38bb64d229d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:08.994619 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9460f20a-6507-49c9-8ea6-38bb64d229d8 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:08.994619 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9460f20a-6507-49c9-8ea6-38bb64d229d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:08.994619 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9460f20a-6507-49c9-8ea6-38bb64d229d8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:09.012036 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9460f20a-6507-49c9-8ea6-38bb64d229d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 09:46:09.019286 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-9460f20a-6507-49c9-8ea6-38bb64d229d8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 02 09:46:09.019286 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:09.019286 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:09.019286 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:09.019286 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:09.019286 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:09.019286 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:09.019286 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:09.019286 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:09.019286 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:09.019286 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:09.019286 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:09.019286 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 09:46:09.019286 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 09:46:09.019286 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 09:46:09.019286 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 09:46:09.021061 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 09:46:09.021061 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 09:46:09.021061 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 09:46:09.021061 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 09:46:09.021061 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:09.021061 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:09.021061 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:09.021061 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:09.021061 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:09.021061 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:09.021061 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:09.021061 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:09.021061 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:09.021061 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:09.021061 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:09.021061 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:09.022643 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:09.022643 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:09.022643 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:09.022643 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:09.022643 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 09:46:09.022643 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 09:46:09.022643 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 09:46:09.022643 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 09:46:09.022643 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 09:46:09.022643 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 09:46:09.022643 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 02 09:46:09.022643 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:46:09.024752 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 192/381] 10.222.0.64 () {56 vars in 1099 bytes} [Tue Aug 2 09:46:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:46:09.032374 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9460f20a-6507-49c9-8ea6-38bb64d229d8 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:09.143323 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4eb3fc-fa2c-4e58-b18e-79cf86db8d34 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:09.156127 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:09.221436 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18089ed5-8b1b-4e8e-b02f-879244662188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:09.222016 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18089ed5-8b1b-4e8e-b02f-879244662188 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:09.222303 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18089ed5-8b1b-4e8e-b02f-879244662188 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:09.222582 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18089ed5-8b1b-4e8e-b02f-879244662188 None admin] PATH_INFO: `/v3/groups/d5a8365c94814c0bb8f8a393e038f939` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:09.238068 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18089ed5-8b1b-4e8e-b02f-879244662188 None admin] RBAC: Authorizing `identity:get_group(group_id=d5a8365c94814c0bb8f8a393e038f939)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:09.239559 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18089ed5-8b1b-4e8e-b02f-879244662188 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:09.241328 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 190/382] 10.222.0.64 () {56 vars in 1180 bytes} [Tue Aug 2 09:46:08 2022] GET /identity/v3/groups/d5a8365c94814c0bb8f8a393e038f939 => generated 229 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:46:09.250066 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18089ed5-8b1b-4e8e-b02f-879244662188 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:09.256776 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9460f20a-6507-49c9-8ea6-38bb64d229d8 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:09.308746 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df72246-897d-4d32-acd4-891446352e41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:09.309384 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df72246-897d-4d32-acd4-891446352e41 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:09.309666 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df72246-897d-4d32-acd4-891446352e41 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:09.309939 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df72246-897d-4d32-acd4-891446352e41 None admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:09.310676 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df72246-897d-4d32-acd4-891446352e41 None admin] RBAC: Adding query filter params (name=member) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:09.310979 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df72246-897d-4d32-acd4-891446352e41 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:09.312406 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df72246-897d-4d32-acd4-891446352e41 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:09.334511 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 193/383] 10.222.0.64 () {56 vars in 1101 bytes} [Tue Aug 2 09:46:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:09.341984 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df72246-897d-4d32-acd4-891446352e41 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:09.425813 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:09.461040 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18089ed5-8b1b-4e8e-b02f-879244662188 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:09.523214 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa9464f-083b-4518-b3ae-18dd1c46243a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:09.523758 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa9464f-083b-4518-b3ae-18dd1c46243a None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:09.524043 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa9464f-083b-4518-b3ae-18dd1c46243a None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:09.524334 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa9464f-083b-4518-b3ae-18dd1c46243a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:09.525144 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa9464f-083b-4518-b3ae-18dd1c46243a None admin] RBAC: Adding query filter params (group.id=d5a8365c94814c0bb8f8a393e038f939, role.id=1000cf318fa14915b8091599eaead5de, scope.project.id=34e3e6719e2a412f926ff9cdc426bf39) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:09.525454 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa9464f-083b-4518-b3ae-18dd1c46243a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:09.526898 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa9464f-083b-4518-b3ae-18dd1c46243a None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:09.533183 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df72246-897d-4d32-acd4-891446352e41 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:09.543687 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 191/384] 10.222.0.64 () {56 vars in 1376 bytes} [Tue Aug 2 09:46:09 2022] GET /identity/v3/role_assignments?group.id=d5a8365c94814c0bb8f8a393e038f939&scope.project.id=34e3e6719e2a412f926ff9cdc426bf39&role.id=1000cf318fa14915b8091599eaead5de => generated 260 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:46:09.552815 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:09.574453 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e090cf1-1c26-4a33-9da7-98e220969a17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:09.575680 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e090cf1-1c26-4a33-9da7-98e220969a17 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:09.576501 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e090cf1-1c26-4a33-9da7-98e220969a17 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:09.576783 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e090cf1-1c26-4a33-9da7-98e220969a17 None admin] PATH_INFO: `/v3/groups/da5d1ad048174277ae2e03e61988bad9` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:09.578158 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e090cf1-1c26-4a33-9da7-98e220969a17 None admin] RBAC: Authorizing `identity:get_group(group_id=da5d1ad048174277ae2e03e61988bad9)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:09.579825 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e090cf1-1c26-4a33-9da7-98e220969a17 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:09.582632 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 194/385] 10.222.0.64 () {56 vars in 1180 bytes} [Tue Aug 2 09:46:09 2022] GET /identity/v3/groups/da5d1ad048174277ae2e03e61988bad9 => generated 226 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:46:09.588993 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa9464f-083b-4518-b3ae-18dd1c46243a None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:09.729724 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa9464f-083b-4518-b3ae-18dd1c46243a None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:09.786195 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c363cb-06c1-4faa-b1e8-af5986605b69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:09.786769 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c363cb-06c1-4faa-b1e8-af5986605b69 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:09.787051 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c363cb-06c1-4faa-b1e8-af5986605b69 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:09.787324 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c363cb-06c1-4faa-b1e8-af5986605b69 None admin] PATH_INFO: `/v3/projects/ae78e3e21c0448c0b412e161cc374652` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:09.793278 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c363cb-06c1-4faa-b1e8-af5986605b69 None admin] RBAC: Authorizing `identity:get_project(project_id=ae78e3e21c0448c0b412e161cc374652)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:09.794791 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c363cb-06c1-4faa-b1e8-af5986605b69 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:09.796996 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 192/386] 10.222.0.64 () {56 vars in 1186 bytes} [Tue Aug 2 09:46:09 2022] GET /identity/v3/projects/ae78e3e21c0448c0b412e161cc374652 => generated 298 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:09.802861 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e090cf1-1c26-4a33-9da7-98e220969a17 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:09.849050 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c041fa67-d5ac-4647-8757-942b00bcc595 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:09.849050 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c041fa67-d5ac-4647-8757-942b00bcc595 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:09.849339 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c041fa67-d5ac-4647-8757-942b00bcc595 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:09.851300 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 193/387] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:46:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 09:46:09.856965 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43151766-d68c-4953-976f-2dc43237b9e2 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:09.857301 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43151766-d68c-4953-976f-2dc43237b9e2 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:09.857546 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43151766-d68c-4953-976f-2dc43237b9e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:09.898935 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-43151766-d68c-4953-976f-2dc43237b9e2 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:46:09.901341 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-43151766-d68c-4953-976f-2dc43237b9e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:46:09.921441 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:09.992187 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 194/388] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:46:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 09:46:10.026689 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e090cf1-1c26-4a33-9da7-98e220969a17 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:10.028754 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43151766-d68c-4953-976f-2dc43237b9e2 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:10.105472 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fa72dc-eb30-4611-8ee3-0992d67f5617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:10.107780 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fa72dc-eb30-4611-8ee3-0992d67f5617 None admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:10.108421 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fa72dc-eb30-4611-8ee3-0992d67f5617 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:10.109066 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fa72dc-eb30-4611-8ee3-0992d67f5617 None admin] PATH_INFO: `/v3/projects/ae78e3e21c0448c0b412e161cc374652/groups/da5d1ad048174277ae2e03e61988bad9/roles/66c1074780954f8e87ed11a5c4e8c36a` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:10.112389 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fa72dc-eb30-4611-8ee3-0992d67f5617 None admin] RBAC: Authorizing `identity:create_grant(project_id=ae78e3e21c0448c0b412e161cc374652, group_id=da5d1ad048174277ae2e03e61988bad9, role_id=66c1074780954f8e87ed11a5c4e8c36a)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:10.118811 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fa72dc-eb30-4611-8ee3-0992d67f5617 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:10.141733 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 195/389] 10.222.0.64 () {58 vars in 1442 bytes} [Tue Aug 2 09:46:09 2022] PUT /identity/v3/projects/ae78e3e21c0448c0b412e161cc374652/groups/da5d1ad048174277ae2e03e61988bad9/roles/66c1074780954f8e87ed11a5c4e8c36a => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:46:10.223273 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43151766-d68c-4953-976f-2dc43237b9e2 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:10.268219 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5db8722-e0ec-45db-8cae-f216e48d7257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:10.268996 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5db8722-e0ec-45db-8cae-f216e48d7257 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:10.269283 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5db8722-e0ec-45db-8cae-f216e48d7257 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:10.269563 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5db8722-e0ec-45db-8cae-f216e48d7257 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:10.281542 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5db8722-e0ec-45db-8cae-f216e48d7257 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 09:46:10.286865 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-f5db8722-e0ec-45db-8cae-f216e48d7257 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 02 09:46:10.286865 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:10.286865 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:10.286865 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:10.286865 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:10.286865 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:10.286865 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:10.286865 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:10.286865 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:10.286865 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:10.286865 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:10.286865 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:10.286865 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 09:46:10.286865 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 09:46:10.286865 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 09:46:10.286865 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 09:46:10.287846 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 09:46:10.287846 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 09:46:10.287846 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 09:46:10.287846 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 09:46:10.287846 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:10.287846 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:10.287846 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:10.287846 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:10.287846 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:10.287846 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:10.287846 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:10.287846 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:10.287846 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:10.287846 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:10.287846 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:10.287846 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:10.288699 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:10.288699 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:10.288699 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:10.288699 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:10.288699 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 09:46:10.288699 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 09:46:10.288699 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 09:46:10.288699 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 09:46:10.288699 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 09:46:10.288699 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 09:46:10.288699 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 02 09:46:10.288699 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:46:10.288699 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 195/390] 10.222.0.64 () {56 vars in 1099 bytes} [Tue Aug 2 09:46:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 09:46:10.294082 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fa72dc-eb30-4611-8ee3-0992d67f5617 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:10.381604 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:10.463665 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fa72dc-eb30-4611-8ee3-0992d67f5617 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:10.502665 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22435d7-1dd0-460b-9157-da76b4103806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:10.503632 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22435d7-1dd0-460b-9157-da76b4103806 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:10.504101 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22435d7-1dd0-460b-9157-da76b4103806 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:10.504595 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22435d7-1dd0-460b-9157-da76b4103806 None admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:10.505534 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22435d7-1dd0-460b-9157-da76b4103806 None admin] RBAC: Adding query filter params (name=member) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:10.505844 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22435d7-1dd0-460b-9157-da76b4103806 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:10.506292 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba60876f-67a0-4dfc-b39f-30b78d7b91d2 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:10.506778 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba60876f-67a0-4dfc-b39f-30b78d7b91d2 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:10.507242 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22435d7-1dd0-460b-9157-da76b4103806 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:10.507336 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba60876f-67a0-4dfc-b39f-30b78d7b91d2 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:10.509154 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 196/391] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:46:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:46:10.516042 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be495061-aef9-46a4-aad0-843846ddb3c0 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:10.516357 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be495061-aef9-46a4-aad0-843846ddb3c0 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:10.516617 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be495061-aef9-46a4-aad0-843846ddb3c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:10.517188 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 196/392] 10.222.0.64 () {56 vars in 1101 bytes} [Tue Aug 2 09:46:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:10.523300 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22435d7-1dd0-460b-9157-da76b4103806 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:10.549521 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:10.565341 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-be495061-aef9-46a4-aad0-843846ddb3c0 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:46:10.566565 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-be495061-aef9-46a4-aad0-843846ddb3c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:46:10.667347 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22435d7-1dd0-460b-9157-da76b4103806 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:10.671632 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:10.723197 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 197/393] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:46:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 09:46:10.742892 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb7f383-28b2-4205-b2cf-b36dc4121344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:10.743561 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb7f383-28b2-4205-b2cf-b36dc4121344 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:10.743908 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb7f383-28b2-4205-b2cf-b36dc4121344 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:10.744195 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb7f383-28b2-4205-b2cf-b36dc4121344 None admin] PATH_INFO: `/v3/groups/da5d1ad048174277ae2e03e61988bad9` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:10.748605 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be495061-aef9-46a4-aad0-843846ddb3c0 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:10.749040 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb7f383-28b2-4205-b2cf-b36dc4121344 None admin] RBAC: Authorizing `identity:get_group(group_id=da5d1ad048174277ae2e03e61988bad9)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:10.750573 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb7f383-28b2-4205-b2cf-b36dc4121344 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:10.753196 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 197/394] 10.222.0.64 () {56 vars in 1180 bytes} [Tue Aug 2 09:46:10 2022] GET /identity/v3/groups/da5d1ad048174277ae2e03e61988bad9 => generated 226 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:46:10.758981 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb7f383-28b2-4205-b2cf-b36dc4121344 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:10.867437 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be495061-aef9-46a4-aad0-843846ddb3c0 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:10.910658 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693bdf3a-8be4-499a-8a0e-77213ac2ccf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:10.911317 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693bdf3a-8be4-499a-8a0e-77213ac2ccf3 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:10.911602 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693bdf3a-8be4-499a-8a0e-77213ac2ccf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:10.911880 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693bdf3a-8be4-499a-8a0e-77213ac2ccf3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:10.914801 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:10.927133 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693bdf3a-8be4-499a-8a0e-77213ac2ccf3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 09:46:10.936957 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-693bdf3a-8be4-499a-8a0e-77213ac2ccf3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 09:46:10.936957 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:10.936957 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:10.936957 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:10.936957 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:10.936957 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:10.936957 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:10.936957 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:10.936957 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:10.936957 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:10.936957 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:10.936957 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:10.936957 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 09:46:10.936957 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 09:46:10.936957 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 09:46:10.936957 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 09:46:10.938017 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 09:46:10.938017 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 09:46:10.938017 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 09:46:10.938017 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 09:46:10.938017 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:10.938017 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:10.938017 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:10.938017 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:10.938017 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:10.938017 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:10.938017 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:10.938017 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:10.938017 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:10.938017 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:10.938017 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:10.938017 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:10.938837 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:10.938837 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:10.938837 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:10.938837 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:10.938837 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 09:46:10.938837 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 09:46:10.938837 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 09:46:10.938837 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 09:46:10.938837 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 09:46:10.938837 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 09:46:10.938837 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 09:46:10.938837 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:46:10.940041 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 198/395] 10.222.0.64 () {56 vars in 1114 bytes} [Tue Aug 2 09:46:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:46:10.948332 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693bdf3a-8be4-499a-8a0e-77213ac2ccf3 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:11.002023 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb7f383-28b2-4205-b2cf-b36dc4121344 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:11.063960 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3395ebb6-25f8-46f2-a69c-60fdd91f22f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:11.067804 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3395ebb6-25f8-46f2-a69c-60fdd91f22f9 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:11.068404 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3395ebb6-25f8-46f2-a69c-60fdd91f22f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:11.068891 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3395ebb6-25f8-46f2-a69c-60fdd91f22f9 None admin] PATH_INFO: `/v3/projects/bc7fa79dde7e42ae92a2a42495efdcc2` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:11.073019 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3395ebb6-25f8-46f2-a69c-60fdd91f22f9 None admin] RBAC: Authorizing `identity:get_project(project_id=bc7fa79dde7e42ae92a2a42495efdcc2)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:11.074788 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3395ebb6-25f8-46f2-a69c-60fdd91f22f9 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:11.077205 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 198/396] 10.222.0.64 () {56 vars in 1186 bytes} [Tue Aug 2 09:46:10 2022] GET /identity/v3/projects/bc7fa79dde7e42ae92a2a42495efdcc2 => generated 302 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:46:11.079127 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693bdf3a-8be4-499a-8a0e-77213ac2ccf3 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:11.084747 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3395ebb6-25f8-46f2-a69c-60fdd91f22f9 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:11.120585 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed59fa6-5ef2-40ac-aeac-87277fb58e64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:11.121209 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed59fa6-5ef2-40ac-aeac-87277fb58e64 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:11.121492 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed59fa6-5ef2-40ac-aeac-87277fb58e64 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:11.121766 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed59fa6-5ef2-40ac-aeac-87277fb58e64 None admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:11.122665 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed59fa6-5ef2-40ac-aeac-87277fb58e64 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:11.122893 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed59fa6-5ef2-40ac-aeac-87277fb58e64 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:11.124329 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed59fa6-5ef2-40ac-aeac-87277fb58e64 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:11.134753 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 199/397] 10.222.0.64 () {56 vars in 1111 bytes} [Tue Aug 2 09:46:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:46:11.140943 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed59fa6-5ef2-40ac-aeac-87277fb58e64 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:11.160870 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:11.211646 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:11.240538 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3395ebb6-25f8-46f2-a69c-60fdd91f22f9 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:11.260726 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed59fa6-5ef2-40ac-aeac-87277fb58e64 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:11.292996 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9020bb5-0898-4d8b-a767-d7cbdd31b520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:11.293708 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9020bb5-0898-4d8b-a767-d7cbdd31b520 None admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:11.294019 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9020bb5-0898-4d8b-a767-d7cbdd31b520 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:11.294281 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9020bb5-0898-4d8b-a767-d7cbdd31b520 None admin] PATH_INFO: `/v3/projects/bc7fa79dde7e42ae92a2a42495efdcc2/groups/da5d1ad048174277ae2e03e61988bad9/roles/66c1074780954f8e87ed11a5c4e8c36a` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:11.298419 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f38bd4-edb1-45f9-ab5e-ca9b2a92389d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:11.299797 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f38bd4-edb1-45f9-ab5e-ca9b2a92389d None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:11.300398 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f38bd4-edb1-45f9-ab5e-ca9b2a92389d None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:11.300841 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f38bd4-edb1-45f9-ab5e-ca9b2a92389d None admin] PATH_INFO: `/v3/groups/da5d1ad048174277ae2e03e61988bad9` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:11.301172 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9020bb5-0898-4d8b-a767-d7cbdd31b520 None admin] RBAC: Authorizing `identity:create_grant(project_id=bc7fa79dde7e42ae92a2a42495efdcc2, group_id=da5d1ad048174277ae2e03e61988bad9, role_id=66c1074780954f8e87ed11a5c4e8c36a)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:11.302381 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f38bd4-edb1-45f9-ab5e-ca9b2a92389d None admin] RBAC: Authorizing `identity:get_group(group_id=da5d1ad048174277ae2e03e61988bad9)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:11.302694 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9020bb5-0898-4d8b-a767-d7cbdd31b520 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:11.304151 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f38bd4-edb1-45f9-ab5e-ca9b2a92389d None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:11.306451 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 200/398] 10.222.0.64 () {56 vars in 1180 bytes} [Tue Aug 2 09:46:11 2022] GET /identity/v3/groups/da5d1ad048174277ae2e03e61988bad9 => generated 226 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:11.312099 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f38bd4-edb1-45f9-ab5e-ca9b2a92389d None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:11.327663 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 199/399] 10.222.0.64 () {58 vars in 1442 bytes} [Tue Aug 2 09:46:11 2022] PUT /identity/v3/projects/bc7fa79dde7e42ae92a2a42495efdcc2/groups/da5d1ad048174277ae2e03e61988bad9/roles/66c1074780954f8e87ed11a5c4e8c36a => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:46:11.407975 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e2550d-ed00-4301-8385-4be0fa698e15 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:11.408906 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e2550d-ed00-4301-8385-4be0fa698e15 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:11.409199 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e2550d-ed00-4301-8385-4be0fa698e15 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:11.410627 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 200/400] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:46:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 09:46:11.418541 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d151b52-362a-4263-8090-2550e86fd649 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:11.418859 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d151b52-362a-4263-8090-2550e86fd649 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:11.419218 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d151b52-362a-4263-8090-2550e86fd649 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:11.447312 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-5d151b52-362a-4263-8090-2550e86fd649 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:46:11.451962 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-5d151b52-362a-4263-8090-2550e86fd649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:46:11.519366 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f38bd4-edb1-45f9-ab5e-ca9b2a92389d None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:11.565677 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 201/401] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:46:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 09:46:11.575615 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:11.584983 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19af045-04cf-416e-bf53-220f6bb62a30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:11.585692 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19af045-04cf-416e-bf53-220f6bb62a30 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:11.586142 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19af045-04cf-416e-bf53-220f6bb62a30 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:11.586528 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19af045-04cf-416e-bf53-220f6bb62a30 None admin] PATH_INFO: `/v3/projects/ae78e3e21c0448c0b412e161cc374652` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:11.588071 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19af045-04cf-416e-bf53-220f6bb62a30 None admin] RBAC: Authorizing `identity:get_project(project_id=ae78e3e21c0448c0b412e161cc374652)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:11.589684 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19af045-04cf-416e-bf53-220f6bb62a30 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:11.591846 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 201/402] 10.222.0.64 () {56 vars in 1186 bytes} [Tue Aug 2 09:46:11 2022] GET /identity/v3/projects/ae78e3e21c0448c0b412e161cc374652 => generated 298 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:46:11.596495 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d151b52-362a-4263-8090-2550e86fd649 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:11.600163 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19af045-04cf-416e-bf53-220f6bb62a30 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:11.632131 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:11.714682 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d151b52-362a-4263-8090-2550e86fd649 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:11.755032 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f142b2-9d73-4269-adac-ebb390fc2ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:11.756053 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f142b2-9d73-4269-adac-ebb390fc2ab8 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:11.756569 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f142b2-9d73-4269-adac-ebb390fc2ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:11.757069 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f142b2-9d73-4269-adac-ebb390fc2ab8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:11.769473 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f142b2-9d73-4269-adac-ebb390fc2ab8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 09:46:11.775286 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-c3f142b2-9d73-4269-adac-ebb390fc2ab8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 09:46:11.775286 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:11.775286 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:11.775286 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:11.775286 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:11.775286 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:11.775286 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:11.775286 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:11.775286 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:11.775286 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:11.775286 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:11.775286 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:11.775286 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 09:46:11.775286 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 09:46:11.775286 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 09:46:11.775286 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 09:46:11.776190 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 09:46:11.776190 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 09:46:11.776190 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 09:46:11.776190 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 09:46:11.776190 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:11.776190 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:11.776190 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:11.776190 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:11.776190 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:11.776190 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:11.776190 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:11.776190 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:11.776190 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:11.776190 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:11.776190 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:11.776190 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:11.777045 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:11.777045 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:11.777045 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:11.777045 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:11.777045 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 09:46:11.777045 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 09:46:11.777045 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 09:46:11.777045 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 09:46:11.777045 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 09:46:11.777045 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 09:46:11.777045 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 09:46:11.777045 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:46:11.779809 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 202/403] 10.222.0.64 () {56 vars in 1114 bytes} [Tue Aug 2 09:46:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:46:11.793127 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f142b2-9d73-4269-adac-ebb390fc2ab8 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:11.878634 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19af045-04cf-416e-bf53-220f6bb62a30 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:11.903467 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:11.915904 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f142b2-9d73-4269-adac-ebb390fc2ab8 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:11.969367 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ba9778-fb98-491b-9fb5-98b854882d8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:11.971342 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ba9778-fb98-491b-9fb5-98b854882d8c None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:11.972118 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ba9778-fb98-491b-9fb5-98b854882d8c None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:11.972902 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ba9778-fb98-491b-9fb5-98b854882d8c None admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:11.974436 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ba9778-fb98-491b-9fb5-98b854882d8c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:11.975249 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ba9778-fb98-491b-9fb5-98b854882d8c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:11.978199 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ba9778-fb98-491b-9fb5-98b854882d8c None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:11.984974 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc098d22-d645-425c-9ef1-161284762c5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:11.985679 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc098d22-d645-425c-9ef1-161284762c5f None admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:11.985971 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc098d22-d645-425c-9ef1-161284762c5f None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:11.986806 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc098d22-d645-425c-9ef1-161284762c5f None admin] PATH_INFO: `/v3/projects/ae78e3e21c0448c0b412e161cc374652/groups/da5d1ad048174277ae2e03e61988bad9/roles/06dfd237063c42a4bff1ad8ca29c07b0` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:11.995351 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 203/404] 10.222.0.64 () {56 vars in 1111 bytes} [Tue Aug 2 09:46:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:46:11.995966 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc098d22-d645-425c-9ef1-161284762c5f None admin] RBAC: Authorizing `identity:create_grant(project_id=ae78e3e21c0448c0b412e161cc374652, group_id=da5d1ad048174277ae2e03e61988bad9, role_id=06dfd237063c42a4bff1ad8ca29c07b0)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:12.001652 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc098d22-d645-425c-9ef1-161284762c5f None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:12.004210 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ba9778-fb98-491b-9fb5-98b854882d8c None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:12.023506 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 202/405] 10.222.0.64 () {58 vars in 1442 bytes} [Tue Aug 2 09:46:11 2022] PUT /identity/v3/projects/ae78e3e21c0448c0b412e161cc374652/groups/da5d1ad048174277ae2e03e61988bad9/roles/06dfd237063c42a4bff1ad8ca29c07b0 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:46:12.164269 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ba9778-fb98-491b-9fb5-98b854882d8c None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:12.226868 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33954e0e-5565-4c71-b5d5-ed4e0f451d27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:12.227534 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33954e0e-5565-4c71-b5d5-ed4e0f451d27 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:12.227814 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33954e0e-5565-4c71-b5d5-ed4e0f451d27 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:12.228084 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33954e0e-5565-4c71-b5d5-ed4e0f451d27 None admin] PATH_INFO: `/v3/groups/da5d1ad048174277ae2e03e61988bad9` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:12.229310 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33954e0e-5565-4c71-b5d5-ed4e0f451d27 None admin] RBAC: Authorizing `identity:get_group(group_id=da5d1ad048174277ae2e03e61988bad9)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:12.230760 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33954e0e-5565-4c71-b5d5-ed4e0f451d27 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:12.232731 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 204/406] 10.222.0.64 () {56 vars in 1180 bytes} [Tue Aug 2 09:46:12 2022] GET /identity/v3/groups/da5d1ad048174277ae2e03e61988bad9 => generated 226 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:12.235252 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca950f12-2ff8-41cb-8844-1cab16439a2a None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:12.236149 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca950f12-2ff8-41cb-8844-1cab16439a2a None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:12.237135 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca950f12-2ff8-41cb-8844-1cab16439a2a None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:12.238880 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 203/407] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:46:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:46:12.240955 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33954e0e-5565-4c71-b5d5-ed4e0f451d27 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:12.245322 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272015fb-5084-4e5d-b2ed-f3d6af5ca867 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:12.245884 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272015fb-5084-4e5d-b2ed-f3d6af5ca867 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:12.246363 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272015fb-5084-4e5d-b2ed-f3d6af5ca867 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:12.278396 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:12.279169 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-272015fb-5084-4e5d-b2ed-f3d6af5ca867 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:46:12.280534 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-272015fb-5084-4e5d-b2ed-f3d6af5ca867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:46:12.355057 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:12.377908 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33954e0e-5565-4c71-b5d5-ed4e0f451d27 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:12.383093 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 204/408] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:46:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 09:46:12.414454 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b7ee18-fa35-4b9e-986a-ea47a875d79e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:12.415114 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b7ee18-fa35-4b9e-986a-ea47a875d79e None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:12.415397 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b7ee18-fa35-4b9e-986a-ea47a875d79e None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:12.415758 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b7ee18-fa35-4b9e-986a-ea47a875d79e None admin] PATH_INFO: `/v3/projects/bc7fa79dde7e42ae92a2a42495efdcc2` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:12.416927 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272015fb-5084-4e5d-b2ed-f3d6af5ca867 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:12.417056 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b7ee18-fa35-4b9e-986a-ea47a875d79e None admin] RBAC: Authorizing `identity:get_project(project_id=bc7fa79dde7e42ae92a2a42495efdcc2)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:12.418506 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b7ee18-fa35-4b9e-986a-ea47a875d79e None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:12.420611 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 205/409] 10.222.0.64 () {56 vars in 1186 bytes} [Tue Aug 2 09:46:12 2022] GET /identity/v3/projects/bc7fa79dde7e42ae92a2a42495efdcc2 => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:12.426607 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b7ee18-fa35-4b9e-986a-ea47a875d79e None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:12.534158 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:12.548212 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b7ee18-fa35-4b9e-986a-ea47a875d79e None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:12.551828 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272015fb-5084-4e5d-b2ed-f3d6af5ca867 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:12.587832 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7c287e-67ea-4120-a41d-5a1dcc218770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:12.588554 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7c287e-67ea-4120-a41d-5a1dcc218770 None admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:12.588839 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7c287e-67ea-4120-a41d-5a1dcc218770 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:12.589109 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7c287e-67ea-4120-a41d-5a1dcc218770 None admin] PATH_INFO: `/v3/projects/bc7fa79dde7e42ae92a2a42495efdcc2/groups/da5d1ad048174277ae2e03e61988bad9/roles/06dfd237063c42a4bff1ad8ca29c07b0` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:12.590953 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f92710d-39ce-4641-aebc-4de62f554775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:12.591316 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7c287e-67ea-4120-a41d-5a1dcc218770 None admin] RBAC: Authorizing `identity:create_grant(project_id=bc7fa79dde7e42ae92a2a42495efdcc2, group_id=da5d1ad048174277ae2e03e61988bad9, role_id=06dfd237063c42a4bff1ad8ca29c07b0)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:12.592036 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f92710d-39ce-4641-aebc-4de62f554775 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:12.592561 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f92710d-39ce-4641-aebc-4de62f554775 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:12.592833 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7c287e-67ea-4120-a41d-5a1dcc218770 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:12.593256 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f92710d-39ce-4641-aebc-4de62f554775 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:12.607932 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f92710d-39ce-4641-aebc-4de62f554775 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 09:46:12.609962 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 206/410] 10.222.0.64 () {58 vars in 1442 bytes} [Tue Aug 2 09:46:12 2022] PUT /identity/v3/projects/bc7fa79dde7e42ae92a2a42495efdcc2/groups/da5d1ad048174277ae2e03e61988bad9/roles/06dfd237063c42a4bff1ad8ca29c07b0 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:46:12.615822 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-6f92710d-39ce-4641-aebc-4de62f554775 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 09:46:12.615822 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:12.615822 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:12.615822 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:12.615822 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:12.615822 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:12.615822 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:12.615822 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:12.615822 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:12.615822 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:12.615822 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:12.615822 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:12.615822 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 09:46:12.615822 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 09:46:12.615822 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 09:46:12.615822 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 09:46:12.617043 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 09:46:12.617043 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 09:46:12.617043 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 09:46:12.617043 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 09:46:12.617043 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:12.617043 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:12.617043 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:12.617043 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:12.617043 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:12.617043 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:12.617043 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:12.617043 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:12.617043 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:12.617043 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:12.617043 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:12.617043 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:12.617899 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:12.617899 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:12.617899 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:12.617899 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:12.617899 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 09:46:12.617899 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 09:46:12.617899 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 09:46:12.617899 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 09:46:12.617899 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 09:46:12.617899 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 09:46:12.617899 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 09:46:12.617899 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:46:12.620940 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 205/411] 10.222.0.64 () {56 vars in 1096 bytes} [Tue Aug 2 09:46:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:46:12.632283 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7c287e-67ea-4120-a41d-5a1dcc218770 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:12.747350 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7c287e-67ea-4120-a41d-5a1dcc218770 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:12.786800 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea9bb49-cdb0-462f-9edd-d57e633b3968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:12.787359 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea9bb49-cdb0-462f-9edd-d57e633b3968 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:12.787648 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea9bb49-cdb0-462f-9edd-d57e633b3968 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:12.787925 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea9bb49-cdb0-462f-9edd-d57e633b3968 None admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:12.788710 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea9bb49-cdb0-462f-9edd-d57e633b3968 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:12.789014 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea9bb49-cdb0-462f-9edd-d57e633b3968 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:12.790438 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea9bb49-cdb0-462f-9edd-d57e633b3968 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:12.800754 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 207/412] 10.222.0.64 () {56 vars in 1099 bytes} [Tue Aug 2 09:46:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:12.806200 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f92710d-39ce-4641-aebc-4de62f554775 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:12.894224 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:12.957534 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5911ec28-9645-4c70-bd9f-7d0c4250d1a1 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:12.958099 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5911ec28-9645-4c70-bd9f-7d0c4250d1a1 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:12.958378 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5911ec28-9645-4c70-bd9f-7d0c4250d1a1 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:12.959933 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 208/413] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:46:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:46:12.966303 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6b73ac-6f6d-4f76-a5af-0f7a1cafb9a4 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:12.966596 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6b73ac-6f6d-4f76-a5af-0f7a1cafb9a4 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:12.966881 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6b73ac-6f6d-4f76-a5af-0f7a1cafb9a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:12.987703 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f92710d-39ce-4641-aebc-4de62f554775 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:12.999948 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-eb6b73ac-6f6d-4f76-a5af-0f7a1cafb9a4 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:46:13.001498 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-eb6b73ac-6f6d-4f76-a5af-0f7a1cafb9a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:46:13.005144 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:13.031989 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4c3ba0-94d6-4cba-b5f2-e60ce6526f73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:13.032895 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4c3ba0-94d6-4cba-b5f2-e60ce6526f73 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:13.033930 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4c3ba0-94d6-4cba-b5f2-e60ce6526f73 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:13.034330 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4c3ba0-94d6-4cba-b5f2-e60ce6526f73 None admin] PATH_INFO: `/v3/groups/d5a8365c94814c0bb8f8a393e038f939` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:13.035767 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4c3ba0-94d6-4cba-b5f2-e60ce6526f73 None admin] RBAC: Authorizing `identity:get_group(group_id=d5a8365c94814c0bb8f8a393e038f939)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:13.037460 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4c3ba0-94d6-4cba-b5f2-e60ce6526f73 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:13.039541 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 206/414] 10.222.0.64 () {56 vars in 1180 bytes} [Tue Aug 2 09:46:12 2022] GET /identity/v3/groups/d5a8365c94814c0bb8f8a393e038f939 => generated 229 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:13.045086 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4c3ba0-94d6-4cba-b5f2-e60ce6526f73 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:13.094790 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 209/415] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:46:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 09:46:13.120596 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6b73ac-6f6d-4f76-a5af-0f7a1cafb9a4 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:13.233795 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6b73ac-6f6d-4f76-a5af-0f7a1cafb9a4 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:13.234612 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4c3ba0-94d6-4cba-b5f2-e60ce6526f73 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:13.266325 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:13.275108 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7a1b29-0413-408a-b91d-8285c5008ae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:13.276099 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7a1b29-0413-408a-b91d-8285c5008ae9 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:13.276629 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7a1b29-0413-408a-b91d-8285c5008ae9 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:13.277141 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7a1b29-0413-408a-b91d-8285c5008ae9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:13.293169 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7a1b29-0413-408a-b91d-8285c5008ae9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 09:46:13.299968 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-5a7a1b29-0413-408a-b91d-8285c5008ae9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 02 09:46:13.299968 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:13.299968 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:13.299968 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:13.299968 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:13.299968 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:13.299968 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:13.299968 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:13.299968 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:13.299968 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:13.299968 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:13.299968 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:13.299968 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 09:46:13.299968 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 09:46:13.299968 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 09:46:13.299968 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 09:46:13.301039 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 09:46:13.301039 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 09:46:13.301039 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 09:46:13.301039 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 09:46:13.301039 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:13.301039 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:13.301039 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:13.301039 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:13.301039 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:13.301039 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:13.301039 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:13.301039 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:13.301039 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:13.301039 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:13.301039 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:13.301039 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:13.301865 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:13.301865 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:13.301865 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:13.301865 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:13.301865 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 09:46:13.301865 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 09:46:13.301865 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 09:46:13.301865 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 09:46:13.301865 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 09:46:13.301865 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 09:46:13.301865 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 02 09:46:13.301865 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:46:13.308721 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:13.313244 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 210/416] 10.222.0.64 () {56 vars in 1099 bytes} [Tue Aug 2 09:46:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:46:13.317158 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7a1b29-0413-408a-b91d-8285c5008ae9 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:13.322829 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f276e7-8122-4fa8-a361-e5dc34f453fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:13.323496 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f276e7-8122-4fa8-a361-e5dc34f453fd None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:13.323785 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f276e7-8122-4fa8-a361-e5dc34f453fd None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:13.324061 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f276e7-8122-4fa8-a361-e5dc34f453fd None admin] PATH_INFO: `/v3/projects/34e3e6719e2a412f926ff9cdc426bf39` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:13.326085 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f276e7-8122-4fa8-a361-e5dc34f453fd None admin] RBAC: Authorizing `identity:get_project(project_id=34e3e6719e2a412f926ff9cdc426bf39)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:13.327563 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f276e7-8122-4fa8-a361-e5dc34f453fd None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:13.329302 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 207/417] 10.222.0.64 () {56 vars in 1186 bytes} [Tue Aug 2 09:46:13 2022] GET /identity/v3/projects/34e3e6719e2a412f926ff9cdc426bf39 => generated 344 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:13.337012 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f276e7-8122-4fa8-a361-e5dc34f453fd None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:13.459635 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7a1b29-0413-408a-b91d-8285c5008ae9 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:13.499782 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefa27a7-66a3-43b1-88f0-ce256696f5f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:13.500444 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefa27a7-66a3-43b1-88f0-ce256696f5f7 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:13.500734 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefa27a7-66a3-43b1-88f0-ce256696f5f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:13.501016 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefa27a7-66a3-43b1-88f0-ce256696f5f7 None admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:13.501810 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefa27a7-66a3-43b1-88f0-ce256696f5f7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:13.502146 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefa27a7-66a3-43b1-88f0-ce256696f5f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:13.503536 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefa27a7-66a3-43b1-88f0-ce256696f5f7 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:13.513158 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 211/418] 10.222.0.64 () {56 vars in 1101 bytes} [Tue Aug 2 09:46:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:13.519278 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefa27a7-66a3-43b1-88f0-ce256696f5f7 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:13.605800 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:13.636775 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefa27a7-66a3-43b1-88f0-ce256696f5f7 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:13.666334 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f276e7-8122-4fa8-a361-e5dc34f453fd None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:13.683181 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e656c0e-bdfa-46cb-9329-fd177fa6941a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:13.683844 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e656c0e-bdfa-46cb-9329-fd177fa6941a None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:13.684122 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e656c0e-bdfa-46cb-9329-fd177fa6941a None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:13.684444 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e656c0e-bdfa-46cb-9329-fd177fa6941a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:13.781342 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e656c0e-bdfa-46cb-9329-fd177fa6941a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:13.784624 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e656c0e-bdfa-46cb-9329-fd177fa6941a None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:13.797260 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf0a973-4a53-484a-9674-78724ac79cbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:13.797966 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf0a973-4a53-484a-9674-78724ac79cbe None admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:13.798256 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf0a973-4a53-484a-9674-78724ac79cbe None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:13.798534 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf0a973-4a53-484a-9674-78724ac79cbe None admin] PATH_INFO: `/v3/projects/34e3e6719e2a412f926ff9cdc426bf39/groups/d5a8365c94814c0bb8f8a393e038f939/roles/1000cf318fa14915b8091599eaead5de` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:13.805183 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf0a973-4a53-484a-9674-78724ac79cbe None admin] RBAC: Authorizing `identity:create_grant(project_id=34e3e6719e2a412f926ff9cdc426bf39, group_id=d5a8365c94814c0bb8f8a393e038f939, role_id=1000cf318fa14915b8091599eaead5de)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:13.806689 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf0a973-4a53-484a-9674-78724ac79cbe None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:13.824187 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 208/419] 10.222.0.64 () {58 vars in 1442 bytes} [Tue Aug 2 09:46:13 2022] PUT /identity/v3/projects/34e3e6719e2a412f926ff9cdc426bf39/groups/d5a8365c94814c0bb8f8a393e038f939/roles/1000cf318fa14915b8091599eaead5de => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:46:13.884999 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:13.929340 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 212/420] 10.222.0.64 () {58 vars in 1282 bytes} [Tue Aug 2 09:46:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 02 09:46:13.935017 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf0a973-4a53-484a-9674-78724ac79cbe None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:13.997049 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:14.091298 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf0a973-4a53-484a-9674-78724ac79cbe None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:14.143490 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3dbe21-f248-4eab-88f9-ec399ed3c007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:14.144158 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3dbe21-f248-4eab-88f9-ec399ed3c007 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:14.144478 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3dbe21-f248-4eab-88f9-ec399ed3c007 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:14.144807 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3dbe21-f248-4eab-88f9-ec399ed3c007 None admin] PATH_INFO: `/v3/projects/ae78e3e21c0448c0b412e161cc374652` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:14.146046 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3dbe21-f248-4eab-88f9-ec399ed3c007 None admin] RBAC: Authorizing `identity:get_project(project_id=ae78e3e21c0448c0b412e161cc374652)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:14.147493 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3dbe21-f248-4eab-88f9-ec399ed3c007 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:14.149533 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 209/421] 10.222.0.64 () {56 vars in 1186 bytes} [Tue Aug 2 09:46:13 2022] GET /identity/v3/projects/ae78e3e21c0448c0b412e161cc374652 => generated 298 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:14.155259 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e656c0e-bdfa-46cb-9329-fd177fa6941a None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:14.229063 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cfb4f5-240a-4cb0-add0-df4b40afa721 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:14.229383 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cfb4f5-240a-4cb0-add0-df4b40afa721 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:14.229625 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cfb4f5-240a-4cb0-add0-df4b40afa721 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:14.231288 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 210/422] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:46:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 09:46:14.241501 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7b45c0-cca6-418e-bdcf-e0758da0f427 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:14.241801 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7b45c0-cca6-418e-bdcf-e0758da0f427 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:14.242065 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7b45c0-cca6-418e-bdcf-e0758da0f427 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:14.262644 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-4d7b45c0-cca6-418e-bdcf-e0758da0f427 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:46:14.268625 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-4d7b45c0-cca6-418e-bdcf-e0758da0f427 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:46:14.282440 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e656c0e-bdfa-46cb-9329-fd177fa6941a None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:14.364160 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae530288-11c0-4b93-ad46-6db834b12a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:14.367738 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae530288-11c0-4b93-ad46-6db834b12a70 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:14.368542 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae530288-11c0-4b93-ad46-6db834b12a70 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:14.370619 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae530288-11c0-4b93-ad46-6db834b12a70 None admin] PATH_INFO: `/v3/groups/da5d1ad048174277ae2e03e61988bad9` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:14.373891 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae530288-11c0-4b93-ad46-6db834b12a70 None admin] RBAC: Authorizing `identity:get_group(group_id=da5d1ad048174277ae2e03e61988bad9)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:14.379538 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae530288-11c0-4b93-ad46-6db834b12a70 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:14.384750 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 213/423] 10.222.0.64 () {56 vars in 1180 bytes} [Tue Aug 2 09:46:14 2022] GET /identity/v3/groups/da5d1ad048174277ae2e03e61988bad9 => generated 226 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:14.395463 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae530288-11c0-4b93-ad46-6db834b12a70 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:14.396931 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 211/424] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:46:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 09:46:14.420556 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7b45c0-cca6-418e-bdcf-e0758da0f427 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:14.447453 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:14.476145 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:14.546638 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7b45c0-cca6-418e-bdcf-e0758da0f427 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:14.569528 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae530288-11c0-4b93-ad46-6db834b12a70 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:14.609341 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e73ae24-c2b8-4fa9-8bb2-301e507991e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:14.609978 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e73ae24-c2b8-4fa9-8bb2-301e507991e6 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:14.610267 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e73ae24-c2b8-4fa9-8bb2-301e507991e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:14.610549 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e73ae24-c2b8-4fa9-8bb2-301e507991e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:14.611596 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65c28b7-fefe-4e99-9b93-d4a89933d614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:14.612337 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65c28b7-fefe-4e99-9b93-d4a89933d614 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:14.612709 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65c28b7-fefe-4e99-9b93-d4a89933d614 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:14.612994 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65c28b7-fefe-4e99-9b93-d4a89933d614 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:14.614114 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e73ae24-c2b8-4fa9-8bb2-301e507991e6 None admin] RBAC: Adding query filter params (group.id=da5d1ad048174277ae2e03e61988bad9, role.id=66c1074780954f8e87ed11a5c4e8c36a, scope.project.id=ae78e3e21c0448c0b412e161cc374652) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:14.614432 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e73ae24-c2b8-4fa9-8bb2-301e507991e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:14.615924 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e73ae24-c2b8-4fa9-8bb2-301e507991e6 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:14.623262 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 214/425] 10.222.0.64 () {56 vars in 1376 bytes} [Tue Aug 2 09:46:14 2022] GET /identity/v3/role_assignments?group.id=da5d1ad048174277ae2e03e61988bad9&scope.project.id=ae78e3e21c0448c0b412e161cc374652&role.id=66c1074780954f8e87ed11a5c4e8c36a => generated 611 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:46:14.643535 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65c28b7-fefe-4e99-9b93-d4a89933d614 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 09:46:14.649375 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-f65c28b7-fefe-4e99-9b93-d4a89933d614 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 02 09:46:14.649375 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:14.649375 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:14.649375 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:14.649375 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:14.649375 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:14.649375 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:14.649375 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:14.649375 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:14.649375 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:14.649375 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:14.649375 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:14.649375 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 09:46:14.649375 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 09:46:14.649375 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 09:46:14.649375 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 09:46:14.650323 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 09:46:14.650323 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 09:46:14.650323 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 09:46:14.650323 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 09:46:14.650323 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:14.650323 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:14.650323 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:14.650323 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:14.650323 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:14.650323 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:14.650323 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:14.650323 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:14.650323 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:14.650323 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:14.650323 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:14.650323 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:14.651187 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:14.651187 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:14.651187 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:14.651187 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:14.651187 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 09:46:14.651187 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 09:46:14.651187 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 09:46:14.651187 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 09:46:14.651187 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 09:46:14.651187 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 09:46:14.651187 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 02 09:46:14.651187 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:46:14.651187 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 212/426] 10.222.0.64 () {56 vars in 1099 bytes} [Tue Aug 2 09:46:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 09:46:14.658477 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e73ae24-c2b8-4fa9-8bb2-301e507991e6 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:14.790897 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:14.801887 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e73ae24-c2b8-4fa9-8bb2-301e507991e6 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:14.864404 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c2ea72-2cb2-49f6-a6d1-99bd5bdcb386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:14.865048 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c2ea72-2cb2-49f6-a6d1-99bd5bdcb386 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:14.865334 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c2ea72-2cb2-49f6-a6d1-99bd5bdcb386 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:14.865610 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c2ea72-2cb2-49f6-a6d1-99bd5bdcb386 None admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:14.866349 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c2ea72-2cb2-49f6-a6d1-99bd5bdcb386 None admin] RBAC: Adding query filter params (name=member) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:14.866659 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c2ea72-2cb2-49f6-a6d1-99bd5bdcb386 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:14.868064 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c2ea72-2cb2-49f6-a6d1-99bd5bdcb386 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:14.882538 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 215/427] 10.222.0.64 () {56 vars in 1101 bytes} [Tue Aug 2 09:46:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:14.891900 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65c28b7-fefe-4e99-9b93-d4a89933d614 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:14.912516 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2596bd5a-47f8-441d-b185-5ab9c3e3ba97 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:14.912730 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2596bd5a-47f8-441d-b185-5ab9c3e3ba97 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:14.913013 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2596bd5a-47f8-441d-b185-5ab9c3e3ba97 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:14.914634 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 216/428] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:46:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 09:46:14.921275 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1524cca6-6d6b-4d9b-969a-2ea8e34e90c7 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:14.921615 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1524cca6-6d6b-4d9b-969a-2ea8e34e90c7 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:14.921833 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1524cca6-6d6b-4d9b-969a-2ea8e34e90c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:14.945678 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-1524cca6-6d6b-4d9b-969a-2ea8e34e90c7 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:46:14.946953 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-1524cca6-6d6b-4d9b-969a-2ea8e34e90c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:46:15.031120 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:15.040512 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65c28b7-fefe-4e99-9b93-d4a89933d614 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:15.057058 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 217/429] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:46:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 09:46:15.082228 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1524cca6-6d6b-4d9b-969a-2ea8e34e90c7 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:15.106366 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11f49f7-09c1-418b-9a5a-333bd11c5fd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:15.107165 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11f49f7-09c1-418b-9a5a-333bd11c5fd0 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:15.107590 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11f49f7-09c1-418b-9a5a-333bd11c5fd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:15.107943 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11f49f7-09c1-418b-9a5a-333bd11c5fd0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:15.186773 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:15.195380 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11f49f7-09c1-418b-9a5a-333bd11c5fd0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:15.197157 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11f49f7-09c1-418b-9a5a-333bd11c5fd0 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:15.206048 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1524cca6-6d6b-4d9b-969a-2ea8e34e90c7 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:15.248190 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204f2c09-e6c4-488a-ac79-b04787ee8f35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:15.249187 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204f2c09-e6c4-488a-ac79-b04787ee8f35 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:15.249748 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204f2c09-e6c4-488a-ac79-b04787ee8f35 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:15.250231 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204f2c09-e6c4-488a-ac79-b04787ee8f35 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:15.263265 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204f2c09-e6c4-488a-ac79-b04787ee8f35 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 09:46:15.268975 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-204f2c09-e6c4-488a-ac79-b04787ee8f35 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 09:46:15.268975 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:15.268975 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:15.268975 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:15.268975 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:15.268975 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:15.268975 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:15.268975 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:15.268975 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:15.268975 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:15.268975 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:15.268975 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:15.268975 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 09:46:15.268975 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 09:46:15.268975 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 09:46:15.268975 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 09:46:15.269914 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 09:46:15.269914 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 09:46:15.269914 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 09:46:15.269914 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 09:46:15.269914 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:15.269914 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:15.269914 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:15.269914 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:15.269914 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:15.269914 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:15.269914 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:15.269914 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:15.269914 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:15.269914 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:15.269914 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:15.269914 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:15.270729 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:15.270729 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:15.270729 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:15.270729 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:15.270729 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 09:46:15.270729 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 09:46:15.270729 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 09:46:15.270729 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 09:46:15.270729 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 09:46:15.270729 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 09:46:15.270729 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 09:46:15.270729 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:46:15.273702 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 218/430] 10.222.0.64 () {56 vars in 1114 bytes} [Tue Aug 2 09:46:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 09:46:15.283057 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204f2c09-e6c4-488a-ac79-b04787ee8f35 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:15.335862 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:15.343825 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 213/431] 10.222.0.64 () {58 vars in 1282 bytes} [Tue Aug 2 09:46:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 09:46:15.345237 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3276e3b4-4377-4c82-a3ff-2379de93b514 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:15.345587 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3276e3b4-4377-4c82-a3ff-2379de93b514 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:15.345868 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3276e3b4-4377-4c82-a3ff-2379de93b514 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:15.347276 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 214/432] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:46:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:46:15.352298 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3276e3b4-4377-4c82-a3ff-2379de93b514 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:15.438750 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204f2c09-e6c4-488a-ac79-b04787ee8f35 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:15.478580 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3a33ff-f03e-4727-8846-3e70de1d7470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:15.479210 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3a33ff-f03e-4727-8846-3e70de1d7470 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:15.479501 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3a33ff-f03e-4727-8846-3e70de1d7470 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:15.479778 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3a33ff-f03e-4727-8846-3e70de1d7470 None admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:15.480538 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3a33ff-f03e-4727-8846-3e70de1d7470 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:15.480847 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3a33ff-f03e-4727-8846-3e70de1d7470 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:15.482230 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3a33ff-f03e-4727-8846-3e70de1d7470 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:15.486974 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3276e3b4-4377-4c82-a3ff-2379de93b514 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:15.490812 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 219/433] 10.222.0.64 () {56 vars in 1111 bytes} [Tue Aug 2 09:46:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:15.492047 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be62bb1d-6cc0-476f-b8fe-4f279e5e8abd None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:15.492370 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be62bb1d-6cc0-476f-b8fe-4f279e5e8abd None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:15.492652 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be62bb1d-6cc0-476f-b8fe-4f279e5e8abd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:15.512822 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-be62bb1d-6cc0-476f-b8fe-4f279e5e8abd None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:46:15.514025 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-be62bb1d-6cc0-476f-b8fe-4f279e5e8abd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:46:15.516428 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:15.532832 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d23556-2ba8-4abf-bf34-db6601058f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:15.533628 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d23556-2ba8-4abf-bf34-db6601058f8a None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:15.533916 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d23556-2ba8-4abf-bf34-db6601058f8a None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:15.534194 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d23556-2ba8-4abf-bf34-db6601058f8a None admin] PATH_INFO: `/v3/projects/bc7fa79dde7e42ae92a2a42495efdcc2` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:15.535366 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d23556-2ba8-4abf-bf34-db6601058f8a None admin] RBAC: Authorizing `identity:get_project(project_id=bc7fa79dde7e42ae92a2a42495efdcc2)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:15.536860 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d23556-2ba8-4abf-bf34-db6601058f8a None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:15.539278 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 215/434] 10.222.0.64 () {56 vars in 1186 bytes} [Tue Aug 2 09:46:15 2022] GET /identity/v3/projects/bc7fa79dde7e42ae92a2a42495efdcc2 => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:15.539617 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d23556-2ba8-4abf-bf34-db6601058f8a None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:15.599713 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 220/435] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:46:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 09:46:15.600829 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be62bb1d-6cc0-476f-b8fe-4f279e5e8abd None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:15.629592 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:15.669137 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d23556-2ba8-4abf-bf34-db6601058f8a None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:15.710288 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902803c5-b863-4d12-84f4-0b583fe161c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:15.710851 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902803c5-b863-4d12-84f4-0b583fe161c6 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:15.711104 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902803c5-b863-4d12-84f4-0b583fe161c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:15.711377 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902803c5-b863-4d12-84f4-0b583fe161c6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:15.718646 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be62bb1d-6cc0-476f-b8fe-4f279e5e8abd None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:15.773562 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957af178-4380-41a6-9f28-5e346ac3b6c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:15.774575 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957af178-4380-41a6-9f28-5e346ac3b6c8 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:15.775034 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957af178-4380-41a6-9f28-5e346ac3b6c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:15.775824 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957af178-4380-41a6-9f28-5e346ac3b6c8 None admin] PATH_INFO: `/v3/groups/da5d1ad048174277ae2e03e61988bad9` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:15.777940 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957af178-4380-41a6-9f28-5e346ac3b6c8 None admin] RBAC: Authorizing `identity:get_group(group_id=da5d1ad048174277ae2e03e61988bad9)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:15.781048 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957af178-4380-41a6-9f28-5e346ac3b6c8 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:15.789699 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 221/436] 10.222.0.64 () {56 vars in 1180 bytes} [Tue Aug 2 09:46:15 2022] GET /identity/v3/groups/da5d1ad048174277ae2e03e61988bad9 => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:15.789699 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957af178-4380-41a6-9f28-5e346ac3b6c8 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:15.800364 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:15.807519 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902803c5-b863-4d12-84f4-0b583fe161c6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:15.809286 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902803c5-b863-4d12-84f4-0b583fe161c6 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:15.892169 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:15.913904 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957af178-4380-41a6-9f28-5e346ac3b6c8 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:15.934603 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 216/437] 10.222.0.64 () {58 vars in 1282 bytes} [Tue Aug 2 09:46:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 09:46:15.937215 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902803c5-b863-4d12-84f4-0b583fe161c6 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:15.956139 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef7ed78-f84a-4984-91b7-1d08cbafe35a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:15.956823 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef7ed78-f84a-4984-91b7-1d08cbafe35a None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:15.957115 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef7ed78-f84a-4984-91b7-1d08cbafe35a None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:15.957393 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef7ed78-f84a-4984-91b7-1d08cbafe35a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:15.969751 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef7ed78-f84a-4984-91b7-1d08cbafe35a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 09:46:15.975346 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-8ef7ed78-f84a-4984-91b7-1d08cbafe35a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 09:46:15.975346 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:15.975346 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:15.975346 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:15.975346 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:15.975346 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:15.975346 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:15.975346 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:15.975346 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:15.975346 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:15.975346 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:15.975346 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:15.975346 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 09:46:15.975346 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 09:46:15.975346 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 09:46:15.975346 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 09:46:15.977699 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 09:46:15.977699 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 09:46:15.977699 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 09:46:15.977699 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 09:46:15.977699 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:15.977699 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:15.977699 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:15.977699 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:15.977699 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:15.977699 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:15.977699 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:15.977699 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:15.977699 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:15.977699 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:15.977699 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:15.977699 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:15.979447 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:15.979447 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:15.979447 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:15.979447 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:15.979447 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 09:46:15.979447 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 09:46:15.979447 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 09:46:15.979447 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 09:46:15.979447 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 09:46:15.979447 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 09:46:15.979447 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 09:46:15.979447 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:46:15.979447 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 222/438] 10.222.0.64 () {56 vars in 1114 bytes} [Tue Aug 2 09:46:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:46:15.979447 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef7ed78-f84a-4984-91b7-1d08cbafe35a None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:16.060596 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:16.079156 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902803c5-b863-4d12-84f4-0b583fe161c6 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:16.113667 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef7ed78-f84a-4984-91b7-1d08cbafe35a None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:16.129705 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3c5333-afcd-43cd-b98a-6c0806b549b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:16.130693 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3c5333-afcd-43cd-b98a-6c0806b549b9 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:16.131104 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3c5333-afcd-43cd-b98a-6c0806b549b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:16.131558 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3c5333-afcd-43cd-b98a-6c0806b549b9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:16.132925 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3c5333-afcd-43cd-b98a-6c0806b549b9 None admin] RBAC: Adding query filter params (group.id=da5d1ad048174277ae2e03e61988bad9, role.id=66c1074780954f8e87ed11a5c4e8c36a, scope.project.id=bc7fa79dde7e42ae92a2a42495efdcc2) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:16.133442 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3c5333-afcd-43cd-b98a-6c0806b549b9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:16.135906 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3c5333-afcd-43cd-b98a-6c0806b549b9 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:16.152293 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 217/439] 10.222.0.64 () {56 vars in 1376 bytes} [Tue Aug 2 09:46:15 2022] GET /identity/v3/role_assignments?group.id=da5d1ad048174277ae2e03e61988bad9&scope.project.id=bc7fa79dde7e42ae92a2a42495efdcc2&role.id=66c1074780954f8e87ed11a5c4e8c36a => generated 611 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:16.152293 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3c5333-afcd-43cd-b98a-6c0806b549b9 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:16.170308 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f3a837-2c7a-4cf8-810c-c31f2fa35f7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:16.171181 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f3a837-2c7a-4cf8-810c-c31f2fa35f7c None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:16.171563 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f3a837-2c7a-4cf8-810c-c31f2fa35f7c None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:16.171947 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f3a837-2c7a-4cf8-810c-c31f2fa35f7c None admin] PATH_INFO: `/v3/projects/ae78e3e21c0448c0b412e161cc374652` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:16.173372 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f3a837-2c7a-4cf8-810c-c31f2fa35f7c None admin] RBAC: Authorizing `identity:get_project(project_id=ae78e3e21c0448c0b412e161cc374652)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:16.175594 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f3a837-2c7a-4cf8-810c-c31f2fa35f7c None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:16.177731 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 223/440] 10.222.0.64 () {56 vars in 1186 bytes} [Tue Aug 2 09:46:15 2022] GET /identity/v3/projects/ae78e3e21c0448c0b412e161cc374652 => generated 298 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:16.183535 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f3a837-2c7a-4cf8-810c-c31f2fa35f7c None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:16.207329 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:16.325635 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3c5333-afcd-43cd-b98a-6c0806b549b9 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:16.359023 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f3a837-2c7a-4cf8-810c-c31f2fa35f7c None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:16.389757 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ae5917-086d-49cd-9dd1-560f03e6e656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:16.390710 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ae5917-086d-49cd-9dd1-560f03e6e656 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:16.391191 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ae5917-086d-49cd-9dd1-560f03e6e656 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:16.391608 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ae5917-086d-49cd-9dd1-560f03e6e656 None admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:16.393630 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ae5917-086d-49cd-9dd1-560f03e6e656 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:16.393630 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ae5917-086d-49cd-9dd1-560f03e6e656 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:16.395513 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ae5917-086d-49cd-9dd1-560f03e6e656 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:16.412435 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 218/441] 10.222.0.64 () {56 vars in 1111 bytes} [Tue Aug 2 09:46:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:16.421281 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f74cb02-c7ec-4630-baa8-326d900fe30e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:16.421522 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ae5917-086d-49cd-9dd1-560f03e6e656 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:16.423604 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f74cb02-c7ec-4630-baa8-326d900fe30e None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:16.424757 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f74cb02-c7ec-4630-baa8-326d900fe30e None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:16.425410 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f74cb02-c7ec-4630-baa8-326d900fe30e None admin] PATH_INFO: `/v3/groups/da5d1ad048174277ae2e03e61988bad9` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:16.427712 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f74cb02-c7ec-4630-baa8-326d900fe30e None admin] RBAC: Authorizing `identity:get_group(group_id=da5d1ad048174277ae2e03e61988bad9)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:16.430323 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f74cb02-c7ec-4630-baa8-326d900fe30e None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:16.438952 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 224/442] 10.222.0.64 () {56 vars in 1180 bytes} [Tue Aug 2 09:46:16 2022] GET /identity/v3/groups/da5d1ad048174277ae2e03e61988bad9 => generated 226 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:16.441997 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f74cb02-c7ec-4630-baa8-326d900fe30e None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:16.446236 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:16.565655 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:16.575987 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ae5917-086d-49cd-9dd1-560f03e6e656 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:16.589040 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f74cb02-c7ec-4630-baa8-326d900fe30e None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:16.611744 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c86350a-3597-4903-bdd3-a9226c0192db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:16.612460 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c86350a-3597-4903-bdd3-a9226c0192db None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:16.612746 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c86350a-3597-4903-bdd3-a9226c0192db None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:16.613021 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c86350a-3597-4903-bdd3-a9226c0192db None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:16.626652 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e9ffd5-9cb7-4e6d-9d7d-2fc10824e236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:16.627769 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e9ffd5-9cb7-4e6d-9d7d-2fc10824e236 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:16.628102 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e9ffd5-9cb7-4e6d-9d7d-2fc10824e236 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:16.628657 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e9ffd5-9cb7-4e6d-9d7d-2fc10824e236 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:16.629244 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e9ffd5-9cb7-4e6d-9d7d-2fc10824e236 None admin] RBAC: Adding query filter params (group.id=da5d1ad048174277ae2e03e61988bad9, role.id=06dfd237063c42a4bff1ad8ca29c07b0, scope.project.id=ae78e3e21c0448c0b412e161cc374652) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:16.629564 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e9ffd5-9cb7-4e6d-9d7d-2fc10824e236 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:16.632081 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e9ffd5-9cb7-4e6d-9d7d-2fc10824e236 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:16.641763 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 225/443] 10.222.0.64 () {56 vars in 1376 bytes} [Tue Aug 2 09:46:16 2022] GET /identity/v3/role_assignments?group.id=da5d1ad048174277ae2e03e61988bad9&scope.project.id=ae78e3e21c0448c0b412e161cc374652&role.id=06dfd237063c42a4bff1ad8ca29c07b0 => generated 611 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:46:16.643074 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8006a52-c76b-46e3-8c65-bd6d9287b6db None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:16.643365 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8006a52-c76b-46e3-8c65-bd6d9287b6db None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:16.643645 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8006a52-c76b-46e3-8c65-bd6d9287b6db None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:16.645474 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 226/444] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:46:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:46:16.652677 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fde112b-6cd5-4b38-8b0e-ee7be9ceb4cf None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:16.653043 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fde112b-6cd5-4b38-8b0e-ee7be9ceb4cf None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:16.653342 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fde112b-6cd5-4b38-8b0e-ee7be9ceb4cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:16.675947 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-3fde112b-6cd5-4b38-8b0e-ee7be9ceb4cf None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:46:16.677229 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-3fde112b-6cd5-4b38-8b0e-ee7be9ceb4cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:46:16.698982 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c86350a-3597-4903-bdd3-a9226c0192db None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:16.700596 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c86350a-3597-4903-bdd3-a9226c0192db None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:16.763428 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:16.773079 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 227/445] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:46:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 09:46:16.796200 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fde112b-6cd5-4b38-8b0e-ee7be9ceb4cf None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:16.813681 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:16.828834 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 219/446] 10.222.0.64 () {58 vars in 1282 bytes} [Tue Aug 2 09:46:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 02 09:46:16.834118 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c86350a-3597-4903-bdd3-a9226c0192db None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:16.933840 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fde112b-6cd5-4b38-8b0e-ee7be9ceb4cf None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:16.972808 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c86350a-3597-4903-bdd3-a9226c0192db None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:16.982163 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3fe580-e743-4a02-83d3-270ed62822e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:16.983059 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3fe580-e743-4a02-83d3-270ed62822e2 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:16.983538 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3fe580-e743-4a02-83d3-270ed62822e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:16.983971 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3fe580-e743-4a02-83d3-270ed62822e2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:17.003148 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3fe580-e743-4a02-83d3-270ed62822e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 09:46:17.008214 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-9a3fe580-e743-4a02-83d3-270ed62822e2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 09:46:17.008214 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:17.008214 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:17.008214 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:17.008214 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:17.008214 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:17.008214 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:17.008214 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:17.008214 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:17.008214 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:17.008214 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:17.008214 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:17.008214 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 09:46:17.008214 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 09:46:17.008214 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 09:46:17.008214 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 09:46:17.009209 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 09:46:17.009209 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 09:46:17.009209 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 09:46:17.009209 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 09:46:17.009209 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:17.009209 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:17.009209 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:17.009209 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:17.009209 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:17.009209 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:17.009209 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:17.009209 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:17.009209 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:17.009209 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:17.009209 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:17.009209 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:17.010042 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:17.010042 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:17.010042 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:17.010042 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:17.010042 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 09:46:17.010042 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 09:46:17.010042 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 09:46:17.010042 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 09:46:17.010042 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 09:46:17.010042 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 09:46:17.010042 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 09:46:17.010042 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:46:17.010042 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 228/447] 10.222.0.64 () {56 vars in 1096 bytes} [Tue Aug 2 09:46:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 09:46:17.017364 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3fe580-e743-4a02-83d3-270ed62822e2 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:17.028751 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96fa40a-2f10-4db2-bb8f-ac593c476811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:17.029792 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96fa40a-2f10-4db2-bb8f-ac593c476811 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:17.031243 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96fa40a-2f10-4db2-bb8f-ac593c476811 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:17.031243 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96fa40a-2f10-4db2-bb8f-ac593c476811 None admin] PATH_INFO: `/v3/projects/bc7fa79dde7e42ae92a2a42495efdcc2` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:17.035407 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96fa40a-2f10-4db2-bb8f-ac593c476811 None admin] RBAC: Authorizing `identity:get_project(project_id=bc7fa79dde7e42ae92a2a42495efdcc2)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:17.038285 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96fa40a-2f10-4db2-bb8f-ac593c476811 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:17.040169 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 220/448] 10.222.0.64 () {56 vars in 1186 bytes} [Tue Aug 2 09:46:16 2022] GET /identity/v3/projects/bc7fa79dde7e42ae92a2a42495efdcc2 => generated 302 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:46:17.045937 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96fa40a-2f10-4db2-bb8f-ac593c476811 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:17.097055 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:17.145927 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:17.158419 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3fe580-e743-4a02-83d3-270ed62822e2 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:17.205355 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96fa40a-2f10-4db2-bb8f-ac593c476811 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:17.212202 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc597f4c-b366-41d0-814c-fe8ecbac6cc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:17.212861 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc597f4c-b366-41d0-814c-fe8ecbac6cc9 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:17.213149 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc597f4c-b366-41d0-814c-fe8ecbac6cc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:17.213434 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc597f4c-b366-41d0-814c-fe8ecbac6cc9 None admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:17.214712 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc597f4c-b366-41d0-814c-fe8ecbac6cc9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:17.215012 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc597f4c-b366-41d0-814c-fe8ecbac6cc9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:17.216837 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc597f4c-b366-41d0-814c-fe8ecbac6cc9 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:17.227527 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 229/449] 10.222.0.64 () {56 vars in 1099 bytes} [Tue Aug 2 09:46:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:46:17.233540 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc597f4c-b366-41d0-814c-fe8ecbac6cc9 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:17.247309 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cede95-7322-4ed8-8407-3dd2cfad77bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:17.247953 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cede95-7322-4ed8-8407-3dd2cfad77bd None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:17.248232 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cede95-7322-4ed8-8407-3dd2cfad77bd None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:17.248533 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cede95-7322-4ed8-8407-3dd2cfad77bd None admin] PATH_INFO: `/v3/groups/da5d1ad048174277ae2e03e61988bad9` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:17.249670 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cede95-7322-4ed8-8407-3dd2cfad77bd None admin] RBAC: Authorizing `identity:get_group(group_id=da5d1ad048174277ae2e03e61988bad9)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:17.251114 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cede95-7322-4ed8-8407-3dd2cfad77bd None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:17.252917 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 221/450] 10.222.0.64 () {56 vars in 1180 bytes} [Tue Aug 2 09:46:17 2022] GET /identity/v3/groups/da5d1ad048174277ae2e03e61988bad9 => generated 226 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:17.258513 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cede95-7322-4ed8-8407-3dd2cfad77bd None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:17.352363 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc597f4c-b366-41d0-814c-fe8ecbac6cc9 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:17.367167 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cede95-7322-4ed8-8407-3dd2cfad77bd None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:17.370040 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:17.388160 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbaf151-c37f-42c8-b1c2-df1876c1c519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:17.388857 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbaf151-c37f-42c8-b1c2-df1876c1c519 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:17.389144 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbaf151-c37f-42c8-b1c2-df1876c1c519 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:17.389428 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbaf151-c37f-42c8-b1c2-df1876c1c519 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:17.397321 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:17.405945 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2decc94d-732a-4d28-8077-81060daeb4ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:17.406573 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2decc94d-732a-4d28-8077-81060daeb4ca None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:17.406858 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2decc94d-732a-4d28-8077-81060daeb4ca None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:17.407134 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2decc94d-732a-4d28-8077-81060daeb4ca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:17.408532 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2decc94d-732a-4d28-8077-81060daeb4ca None admin] RBAC: Adding query filter params (group.id=da5d1ad048174277ae2e03e61988bad9, role.id=06dfd237063c42a4bff1ad8ca29c07b0, scope.project.id=bc7fa79dde7e42ae92a2a42495efdcc2) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:17.408855 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2decc94d-732a-4d28-8077-81060daeb4ca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:17.410301 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2decc94d-732a-4d28-8077-81060daeb4ca None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:17.418221 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 222/451] 10.222.0.64 () {56 vars in 1376 bytes} [Tue Aug 2 09:46:17 2022] GET /identity/v3/role_assignments?group.id=da5d1ad048174277ae2e03e61988bad9&scope.project.id=bc7fa79dde7e42ae92a2a42495efdcc2&role.id=06dfd237063c42a4bff1ad8ca29c07b0 => generated 611 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:46:17.467780 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbaf151-c37f-42c8-b1c2-df1876c1c519 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:17.469324 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbaf151-c37f-42c8-b1c2-df1876c1c519 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:17.581071 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 230/452] 10.222.0.64 () {58 vars in 1282 bytes} [Tue Aug 2 09:46:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 02 09:46:17.586755 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2decc94d-732a-4d28-8077-81060daeb4ca None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:17.701676 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2decc94d-732a-4d28-8077-81060daeb4ca None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:17.741621 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2963066-fa6a-4b22-861c-a5e2f19123af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:17.742283 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2963066-fa6a-4b22-861c-a5e2f19123af None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:17.742569 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2963066-fa6a-4b22-861c-a5e2f19123af None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:17.742852 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2963066-fa6a-4b22-861c-a5e2f19123af None admin] PATH_INFO: `/v3/projects/34e3e6719e2a412f926ff9cdc426bf39` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:17.744043 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2963066-fa6a-4b22-861c-a5e2f19123af None admin] RBAC: Authorizing `identity:get_project(project_id=34e3e6719e2a412f926ff9cdc426bf39)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:17.745580 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2963066-fa6a-4b22-861c-a5e2f19123af None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:17.747291 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 223/453] 10.222.0.64 () {56 vars in 1186 bytes} [Tue Aug 2 09:46:17 2022] GET /identity/v3/projects/34e3e6719e2a412f926ff9cdc426bf39 => generated 344 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:17.753700 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbaf151-c37f-42c8-b1c2-df1876c1c519 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:17.773017 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:17.860899 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbaf151-c37f-42c8-b1c2-df1876c1c519 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:17.897568 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16c6f80-ff64-46f2-8f36-b915591c6797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:17.898230 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16c6f80-ff64-46f2-8f36-b915591c6797 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:17.898516 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16c6f80-ff64-46f2-8f36-b915591c6797 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:17.898799 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16c6f80-ff64-46f2-8f36-b915591c6797 None admin] PATH_INFO: `/v3/groups/d5a8365c94814c0bb8f8a393e038f939` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:17.899969 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16c6f80-ff64-46f2-8f36-b915591c6797 None admin] RBAC: Authorizing `identity:get_group(group_id=d5a8365c94814c0bb8f8a393e038f939)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:17.901480 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16c6f80-ff64-46f2-8f36-b915591c6797 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:17.903108 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 231/454] 10.222.0.64 () {56 vars in 1180 bytes} [Tue Aug 2 09:46:17 2022] GET /identity/v3/groups/d5a8365c94814c0bb8f8a393e038f939 => generated 229 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:17.909202 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2963066-fa6a-4b22-861c-a5e2f19123af None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:17.970507 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:18.012402 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2963066-fa6a-4b22-861c-a5e2f19123af None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:18.046141 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b33991-c501-4d44-b0a8-e655f2e63e59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:18.046762 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b33991-c501-4d44-b0a8-e655f2e63e59 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:18.047046 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b33991-c501-4d44-b0a8-e655f2e63e59 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:18.047325 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b33991-c501-4d44-b0a8-e655f2e63e59 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:18.048130 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b33991-c501-4d44-b0a8-e655f2e63e59 None admin] RBAC: Adding query filter params (group.id=d5a8365c94814c0bb8f8a393e038f939, role.id=1000cf318fa14915b8091599eaead5de, scope.project.id=34e3e6719e2a412f926ff9cdc426bf39) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:18.048997 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b33991-c501-4d44-b0a8-e655f2e63e59 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:18.050434 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b33991-c501-4d44-b0a8-e655f2e63e59 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:18.056305 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 224/455] 10.222.0.64 () {56 vars in 1376 bytes} [Tue Aug 2 09:46:17 2022] GET /identity/v3/role_assignments?group.id=d5a8365c94814c0bb8f8a393e038f939&scope.project.id=34e3e6719e2a412f926ff9cdc426bf39&role.id=1000cf318fa14915b8091599eaead5de => generated 611 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:46:20.898192 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62c121c-9f2c-42dd-8b29-14eeb1d11832 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:20.898715 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62c121c-9f2c-42dd-8b29-14eeb1d11832 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:20.898812 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62c121c-9f2c-42dd-8b29-14eeb1d11832 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:20.900584 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 232/456] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:46:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:46:20.905350 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400dcbd9-a6c9-4d29-9d64-e10d07734617 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:20.905350 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400dcbd9-a6c9-4d29-9d64-e10d07734617 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:20.905350 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400dcbd9-a6c9-4d29-9d64-e10d07734617 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:20.907023 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 225/457] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:46:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:46:20.910864 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fdee08-a391-4939-9f1e-f47a66902fa3 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:20.911440 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fdee08-a391-4939-9f1e-f47a66902fa3 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:20.911921 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fdee08-a391-4939-9f1e-f47a66902fa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:20.917950 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f15da5-d581-4076-a35f-7c47167d77e9 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:20.918514 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f15da5-d581-4076-a35f-7c47167d77e9 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:20.918994 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f15da5-d581-4076-a35f-7c47167d77e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:20.950050 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-34f15da5-d581-4076-a35f-7c47167d77e9 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:46:20.954522 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-34f15da5-d581-4076-a35f-7c47167d77e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:46:20.956259 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-35fdee08-a391-4939-9f1e-f47a66902fa3 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:46:20.957681 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-35fdee08-a391-4939-9f1e-f47a66902fa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:46:21.086089 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 226/458] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:46:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 02 09:46:21.103285 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 233/459] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:46:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 09:46:21.111263 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b53206-97d6-4603-84b7-6f5523af06b0 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:21.111556 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b53206-97d6-4603-84b7-6f5523af06b0 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:21.111828 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b53206-97d6-4603-84b7-6f5523af06b0 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:21.113080 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fdee08-a391-4939-9f1e-f47a66902fa3 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:21.113482 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 227/460] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:46:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:46:21.120100 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c71f1d3-69a9-4169-8e5a-cb1ab7f02a72 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:21.120447 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c71f1d3-69a9-4169-8e5a-cb1ab7f02a72 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:21.120723 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c71f1d3-69a9-4169-8e5a-cb1ab7f02a72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:21.143273 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:21.144289 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-3c71f1d3-69a9-4169-8e5a-cb1ab7f02a72 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:46:21.145494 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-3c71f1d3-69a9-4169-8e5a-cb1ab7f02a72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:46:21.208725 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:21.234335 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 228/461] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:46:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 09:46:21.240343 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c71f1d3-69a9-4169-8e5a-cb1ab7f02a72 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:21.271829 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fdee08-a391-4939-9f1e-f47a66902fa3 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:21.314031 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8768c6a-0222-460c-b87b-1b97b0b123b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:21.315173 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8768c6a-0222-460c-b87b-1b97b0b123b3 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:21.315458 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8768c6a-0222-460c-b87b-1b97b0b123b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:21.315737 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8768c6a-0222-460c-b87b-1b97b0b123b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:21.331015 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8768c6a-0222-460c-b87b-1b97b0b123b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:21.333438 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8768c6a-0222-460c-b87b-1b97b0b123b3 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:21.354289 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-b8768c6a-0222-460c-b87b-1b97b0b123b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:46:21.354289 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:21.354289 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:21.354289 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:21.354289 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:21.354289 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:21.354289 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:21.354289 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:21.354289 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:21.354289 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:21.354289 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:21.354289 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:21.354289 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 09:46:21.354289 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 09:46:21.354289 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 09:46:21.355270 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 09:46:21.355270 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:21.355270 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:21.355270 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:21.355270 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:21.355270 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:21.355270 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:21.355270 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:21.355270 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:21.355270 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:21.355270 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:21.355270 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:21.355270 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:21.355270 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:21.355270 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:21.355270 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:21.355270 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:21.361795 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 09:46:21.361795 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 09:46:21.361795 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:46:21.361795 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:46:21.361795 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 234/462] 10.222.0.64 () {56 vars in 1108 bytes} [Tue Aug 2 09:46:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 09:46:21.361795 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c8a17f-9970-4766-b702-14c6db50e46b None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:21.361795 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c8a17f-9970-4766-b702-14c6db50e46b None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:21.361795 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c8a17f-9970-4766-b702-14c6db50e46b None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:21.361795 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 235/463] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:46:21 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:46:21.362421 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c8a17f-9970-4766-b702-14c6db50e46b None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:21.380148 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c71f1d3-69a9-4169-8e5a-cb1ab7f02a72 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:21.425673 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50a91eb-1fce-44ef-980c-26b91c31ff7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:21.426415 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50a91eb-1fce-44ef-980c-26b91c31ff7f None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:21.426703 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50a91eb-1fce-44ef-980c-26b91c31ff7f None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:21.426991 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50a91eb-1fce-44ef-980c-26b91c31ff7f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:21.439202 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50a91eb-1fce-44ef-980c-26b91c31ff7f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:21.439586 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:21.441453 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50a91eb-1fce-44ef-980c-26b91c31ff7f None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:21.461331 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-c50a91eb-1fce-44ef-980c-26b91c31ff7f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:46:21.461331 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:21.461331 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:21.461331 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:21.461331 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:21.461331 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:21.461331 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:21.461331 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:21.461331 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:21.461331 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:21.461331 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:21.461331 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:21.461331 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 09:46:21.461331 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 09:46:21.461331 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 09:46:21.462249 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 09:46:21.462249 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:21.462249 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:21.462249 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:21.462249 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:21.462249 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:21.462249 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:21.462249 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:21.462249 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:21.462249 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:21.462249 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:21.462249 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:21.462249 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:21.462249 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:21.462249 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:21.462249 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:21.462249 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:21.468443 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 09:46:21.468443 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 09:46:21.468443 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:46:21.468443 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:46:21.468443 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 229/464] 10.222.0.64 () {56 vars in 1108 bytes} [Tue Aug 2 09:46:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:46:21.468443 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50a91eb-1fce-44ef-980c-26b91c31ff7f None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:21.483314 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c8a17f-9970-4766-b702-14c6db50e46b None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:21.509326 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:21.524017 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdcbb41-dee9-41ff-9d4a-37314f2012bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:21.524692 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdcbb41-dee9-41ff-9d4a-37314f2012bd None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:21.524978 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdcbb41-dee9-41ff-9d4a-37314f2012bd None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:21.525253 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdcbb41-dee9-41ff-9d4a-37314f2012bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:21.537959 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdcbb41-dee9-41ff-9d4a-37314f2012bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:21.539455 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdcbb41-dee9-41ff-9d4a-37314f2012bd None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:21.554791 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-cfdcbb41-dee9-41ff-9d4a-37314f2012bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:46:21.554791 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:21.554791 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:21.554791 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:21.554791 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:21.554791 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:21.554791 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:21.554791 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:21.554791 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:21.554791 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:21.554791 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:21.554791 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:21.554791 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 09:46:21.554791 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 09:46:21.554791 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 09:46:21.555821 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 09:46:21.555821 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:21.555821 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:21.555821 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:21.555821 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:21.555821 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:21.555821 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:21.555821 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:21.555821 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:21.555821 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:21.555821 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:21.555821 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:21.555821 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:21.555821 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:21.555821 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:21.555821 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:21.555821 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:21.556724 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 09:46:21.556724 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 09:46:21.556724 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:46:21.556724 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:46:21.556724 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 236/465] 10.222.0.64 () {56 vars in 1108 bytes} [Tue Aug 2 09:46:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:46:21.557338 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6031ccd9-d7e8-4c62-8227-154536709970 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:21.557502 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6031ccd9-d7e8-4c62-8227-154536709970 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:21.557779 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6031ccd9-d7e8-4c62-8227-154536709970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:21.576564 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-6031ccd9-d7e8-4c62-8227-154536709970 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:46:21.577790 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-6031ccd9-d7e8-4c62-8227-154536709970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:46:21.598446 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50a91eb-1fce-44ef-980c-26b91c31ff7f None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:21.644961 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c66c00a-988d-40f0-b7e9-404a50719cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:21.645694 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c66c00a-988d-40f0-b7e9-404a50719cd2 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:21.645984 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c66c00a-988d-40f0-b7e9-404a50719cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:21.646263 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c66c00a-988d-40f0-b7e9-404a50719cd2 None admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:21.646992 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c66c00a-988d-40f0-b7e9-404a50719cd2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:21.647377 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c66c00a-988d-40f0-b7e9-404a50719cd2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:21.648784 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c66c00a-988d-40f0-b7e9-404a50719cd2 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:21.675857 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 230/466] 10.222.0.64 () {56 vars in 1109 bytes} [Tue Aug 2 09:46:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:21.675857 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c66c00a-988d-40f0-b7e9-404a50719cd2 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:21.687958 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 237/467] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:46:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 09:46:21.687958 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6031ccd9-d7e8-4c62-8227-154536709970 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:21.760445 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:21.789310 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:21.795997 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c66c00a-988d-40f0-b7e9-404a50719cd2 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:21.830959 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6031ccd9-d7e8-4c62-8227-154536709970 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:21.842863 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a579ed62-e563-4267-8520-ea3729783880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:21.843669 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a579ed62-e563-4267-8520-ea3729783880 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:21.844156 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a579ed62-e563-4267-8520-ea3729783880 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:21.844623 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a579ed62-e563-4267-8520-ea3729783880 None admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:21.845883 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a579ed62-e563-4267-8520-ea3729783880 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:21.846433 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a579ed62-e563-4267-8520-ea3729783880 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:21.849080 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a579ed62-e563-4267-8520-ea3729783880 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:21.865798 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 231/468] 10.222.0.64 () {56 vars in 1109 bytes} [Tue Aug 2 09:46:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:21.866487 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a579ed62-e563-4267-8520-ea3729783880 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:21.878086 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4eebec-a905-4e4b-b231-7fa1c130e2b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:21.878852 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4eebec-a905-4e4b-b231-7fa1c130e2b1 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:21.879142 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4eebec-a905-4e4b-b231-7fa1c130e2b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:21.879419 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4eebec-a905-4e4b-b231-7fa1c130e2b1 None admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:21.880343 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4eebec-a905-4e4b-b231-7fa1c130e2b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:21.880461 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4eebec-a905-4e4b-b231-7fa1c130e2b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:21.882044 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4eebec-a905-4e4b-b231-7fa1c130e2b1 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:21.896474 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 238/469] 10.222.0.64 () {56 vars in 1109 bytes} [Tue Aug 2 09:46:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:21.901767 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4eebec-a905-4e4b-b231-7fa1c130e2b1 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:21.997265 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a579ed62-e563-4267-8520-ea3729783880 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:22.017514 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4eebec-a905-4e4b-b231-7fa1c130e2b1 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:22.035082 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:22.059956 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff35243e-d232-4663-b63e-347a33be016d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:22.060616 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff35243e-d232-4663-b63e-347a33be016d None admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:22.061010 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff35243e-d232-4663-b63e-347a33be016d None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:22.061358 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff35243e-d232-4663-b63e-347a33be016d None admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:22.062162 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff35243e-d232-4663-b63e-347a33be016d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:22.063643 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff35243e-d232-4663-b63e-347a33be016d None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:22.074029 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186e4859-fbda-4f6b-91dc-399c044d23ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:22.074956 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186e4859-fbda-4f6b-91dc-399c044d23ad None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:22.075373 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186e4859-fbda-4f6b-91dc-399c044d23ad None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:22.075790 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186e4859-fbda-4f6b-91dc-399c044d23ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:22.090147 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 232/470] 10.222.0.64 () {60 vars in 1132 bytes} [Tue Aug 2 09:46:21 2022] POST /identity/v3/users => generated 246 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:46:22.090639 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186e4859-fbda-4f6b-91dc-399c044d23ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:22.091029 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff35243e-d232-4663-b63e-347a33be016d None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:22.092676 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186e4859-fbda-4f6b-91dc-399c044d23ad None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:22.122714 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:22.123420 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-186e4859-fbda-4f6b-91dc-399c044d23ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:46:22.123420 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:22.123420 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:22.123420 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:22.123420 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:22.123420 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:22.123420 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:22.123420 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:22.123420 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:22.123420 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:22.123420 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:22.123420 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:22.123420 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 09:46:22.123420 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 09:46:22.123420 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 09:46:22.124604 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 09:46:22.124604 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:22.124604 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:22.124604 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:22.124604 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:22.124604 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:22.124604 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:22.124604 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:22.124604 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:22.124604 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:22.124604 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:22.124604 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:22.124604 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:22.124604 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:22.124604 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:22.124604 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:22.124604 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:22.126190 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 09:46:22.126190 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 09:46:22.126190 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:46:22.126190 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:46:22.129530 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 239/471] 10.222.0.64 () {56 vars in 1108 bytes} [Tue Aug 2 09:46:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:46:22.131225 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186e4859-fbda-4f6b-91dc-399c044d23ad None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:22.250462 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff35243e-d232-4663-b63e-347a33be016d None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:22.288239 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186e4859-fbda-4f6b-91dc-399c044d23ad None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:22.296096 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f755dec-6dc0-4f98-b464-64e39a680ac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:22.296796 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f755dec-6dc0-4f98-b464-64e39a680ac3 None admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:22.297092 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f755dec-6dc0-4f98-b464-64e39a680ac3 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:22.297375 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f755dec-6dc0-4f98-b464-64e39a680ac3 None admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:22.298084 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f755dec-6dc0-4f98-b464-64e39a680ac3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:22.300162 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f755dec-6dc0-4f98-b464-64e39a680ac3 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:22.331350 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48da524a-f7cf-4271-bc68-63654d9a2081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:22.331926 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 233/472] 10.222.0.64 () {60 vars in 1132 bytes} [Tue Aug 2 09:46:22 2022] POST /identity/v3/users => generated 248 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:46:22.332078 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48da524a-f7cf-4271-bc68-63654d9a2081 None admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:22.332359 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48da524a-f7cf-4271-bc68-63654d9a2081 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:22.332583 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48da524a-f7cf-4271-bc68-63654d9a2081 None admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:22.332860 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f755dec-6dc0-4f98-b464-64e39a680ac3 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:22.333254 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48da524a-f7cf-4271-bc68-63654d9a2081 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:22.334792 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48da524a-f7cf-4271-bc68-63654d9a2081 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:22.368936 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 240/473] 10.222.0.64 () {60 vars in 1132 bytes} [Tue Aug 2 09:46:22 2022] POST /identity/v3/users => generated 248 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:46:22.370687 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f25db84-7ce5-4623-baec-4419cdad2583 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:22.371215 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f25db84-7ce5-4623-baec-4419cdad2583 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:22.371771 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f25db84-7ce5-4623-baec-4419cdad2583 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:22.373607 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 241/474] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:46:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:46:22.383855 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d453b240-49f5-4f4f-87ee-e163598e8ce4 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:22.384598 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d453b240-49f5-4f4f-87ee-e163598e8ce4 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:22.385132 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d453b240-49f5-4f4f-87ee-e163598e8ce4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:22.405658 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-d453b240-49f5-4f4f-87ee-e163598e8ce4 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:46:22.407440 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-d453b240-49f5-4f4f-87ee-e163598e8ce4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:46:22.425314 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:22.448706 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:22.496675 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 242/475] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:46:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 09:46:22.536704 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f755dec-6dc0-4f98-b464-64e39a680ac3 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:22.541102 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d453b240-49f5-4f4f-87ee-e163598e8ce4 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:22.596062 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0bdf59-5248-44b4-968a-062464d61d76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:22.597313 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0bdf59-5248-44b4-968a-062464d61d76 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:22.597313 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0bdf59-5248-44b4-968a-062464d61d76 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:22.597479 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0bdf59-5248-44b4-968a-062464d61d76 None admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:22.598944 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0bdf59-5248-44b4-968a-062464d61d76 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:22.599148 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0bdf59-5248-44b4-968a-062464d61d76 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:22.601117 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0bdf59-5248-44b4-968a-062464d61d76 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:22.616771 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 234/476] 10.222.0.64 () {56 vars in 1109 bytes} [Tue Aug 2 09:46:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:22.622307 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0bdf59-5248-44b4-968a-062464d61d76 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:22.684699 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d453b240-49f5-4f4f-87ee-e163598e8ce4 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:22.729534 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d4fea6-8587-42a4-8009-ec61a379111a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:22.730178 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d4fea6-8587-42a4-8009-ec61a379111a None admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:22.730458 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d4fea6-8587-42a4-8009-ec61a379111a None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:22.730730 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d4fea6-8587-42a4-8009-ec61a379111a None admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:22.731471 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d4fea6-8587-42a4-8009-ec61a379111a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:22.732955 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d4fea6-8587-42a4-8009-ec61a379111a None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:22.747035 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.sql.core [None req-90d4fea6-8587-42a4-8009-ec61a379111a None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 02 09:46:22.747035 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 02 09:46:22.747035 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [parameters: {'id': '732eced04643490dab758216a51d6a8c', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 02 09:46:22.747035 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80153) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 02 09:46:22.765095 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0bdf59-5248-44b4-968a-062464d61d76 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:22.770520 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-90d4fea6-8587-42a4-8009-ec61a379111a None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 02 09:46:22.770520 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:22.770520 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 02 09:46:22.770520 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 02 09:46:22.770520 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 02 09:46:22.770520 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 02 09:46:22.770520 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 02 09:46:22.770520 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application result = self._query(query) Aug 02 09:46:22.770520 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 02 09:46:22.770520 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application conn.query(q) Aug 02 09:46:22.770520 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 02 09:46:22.770520 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 02 09:46:22.770520 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 02 09:46:22.770520 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application result.read() Aug 02 09:46:22.770520 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 02 09:46:22.771434 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 02 09:46:22.771434 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 02 09:46:22.771434 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 02 09:46:22.771434 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 02 09:46:22.771434 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 02 09:46:22.771434 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 02 09:46:22.771434 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 02 09:46:22.771434 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 02 09:46:22.771434 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:46:22.771434 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 02 09:46:22.771434 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:46:22.771434 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:22.771434 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 02 09:46:22.771434 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 02 09:46:22.771434 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 02 09:46:22.771434 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return ref.to_dict() Aug 02 09:46:22.771434 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 02 09:46:22.771434 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application next(self.gen) Aug 02 09:46:22.772465 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 02 09:46:22.772465 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application yield resource Aug 02 09:46:22.772465 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 02 09:46:22.772465 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application next(self.gen) Aug 02 09:46:22.772465 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 02 09:46:22.772465 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application self.session.rollback() Aug 02 09:46:22.772465 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 02 09:46:22.772465 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application self.force_reraise() Aug 02 09:46:22.772465 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 02 09:46:22.772465 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise self.value Aug 02 09:46:22.772465 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 02 09:46:22.772465 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 02 09:46:22.772465 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 02 09:46:22.772465 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application session.commit() Aug 02 09:46:22.772465 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 02 09:46:22.772465 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 02 09:46:22.772465 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 02 09:46:22.773345 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application self._prepare_impl() Aug 02 09:46:22.773345 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 02 09:46:22.773345 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application self.session.flush() Aug 02 09:46:22.773345 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 02 09:46:22.773345 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application self._flush(objects) Aug 02 09:46:22.773345 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 02 09:46:22.773345 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 02 09:46:22.773345 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 02 09:46:22.773345 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application compat.raise_( Aug 02 09:46:22.773345 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 02 09:46:22.773345 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception Aug 02 09:46:22.773345 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 02 09:46:22.773345 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application flush_context.execute() Aug 02 09:46:22.773345 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 02 09:46:22.773345 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rec.execute(self) Aug 02 09:46:22.773345 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 02 09:46:22.773345 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 02 09:46:22.774266 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 02 09:46:22.774266 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 02 09:46:22.774266 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 02 09:46:22.774266 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 02 09:46:22.774266 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 02 09:46:22.774266 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 02 09:46:22.774266 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 02 09:46:22.774266 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 02 09:46:22.774266 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 02 09:46:22.774266 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 02 09:46:22.774266 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 02 09:46:22.774266 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 02 09:46:22.774266 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 02 09:46:22.774266 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 02 09:46:22.774266 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 02 09:46:22.774266 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception Aug 02 09:46:22.775093 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 02 09:46:22.775093 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 02 09:46:22.775093 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 02 09:46:22.775093 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 02 09:46:22.775093 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 02 09:46:22.775093 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application result = self._query(query) Aug 02 09:46:22.775093 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 02 09:46:22.775093 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application conn.query(q) Aug 02 09:46:22.775093 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 02 09:46:22.775093 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 02 09:46:22.775093 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 02 09:46:22.775093 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application result.read() Aug 02 09:46:22.775093 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 02 09:46:22.775093 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 02 09:46:22.775093 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 02 09:46:22.775093 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 02 09:46:22.775899 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 02 09:46:22.775899 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 02 09:46:22.775899 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 02 09:46:22.775899 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 02 09:46:22.775899 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 02 09:46:22.775899 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 02 09:46:22.775899 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [parameters: {'id': '732eced04643490dab758216a51d6a8c', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 02 09:46:22.775899 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 02 09:46:22.775899 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:46:22.775899 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 09:46:22.775899 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:46:22.775899 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:22.775899 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:22.775899 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:22.775899 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:22.775899 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:22.778338 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:22.778338 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:22.778338 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:22.778338 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:22.778338 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:22.778338 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:22.778338 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 02 09:46:22.778338 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 02 09:46:22.778338 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:22.778338 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:22.778338 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 02 09:46:22.778338 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 02 09:46:22.778338 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 02 09:46:22.778338 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 02 09:46:22.778338 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 02 09:46:22.778338 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:46:22.780389 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 243/477] 10.222.0.64 () {60 vars in 1131 bytes} [Tue Aug 2 09:46:22 2022] POST /identity/v3/roles => generated 144 bytes in 231 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 02 09:46:22.790879 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d4fea6-8587-42a4-8009-ec61a379111a None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:22.809366 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:22.815045 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:22.818554 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b227ef-ee11-4f3e-ab69-5f14b0be8b5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:22.820313 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b227ef-ee11-4f3e-ab69-5f14b0be8b5e None admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:22.820845 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b227ef-ee11-4f3e-ab69-5f14b0be8b5e None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:22.821422 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b227ef-ee11-4f3e-ab69-5f14b0be8b5e None admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:22.822340 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b227ef-ee11-4f3e-ab69-5f14b0be8b5e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:22.824185 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b227ef-ee11-4f3e-ab69-5f14b0be8b5e None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:22.866648 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 235/478] 10.222.0.64 () {60 vars in 1132 bytes} [Tue Aug 2 09:46:22 2022] POST /identity/v3/users => generated 249 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 09:46:22.935486 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d4fea6-8587-42a4-8009-ec61a379111a None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:22.977858 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d433a94c-93d0-44cb-b7d5-0400cbf130a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:22.978531 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d433a94c-93d0-44cb-b7d5-0400cbf130a9 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:22.978815 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d433a94c-93d0-44cb-b7d5-0400cbf130a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:22.979090 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d433a94c-93d0-44cb-b7d5-0400cbf130a9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:22.992004 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d433a94c-93d0-44cb-b7d5-0400cbf130a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 09:46:22.997218 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-d433a94c-93d0-44cb-b7d5-0400cbf130a9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 09:46:22.997218 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:22.997218 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:22.997218 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:22.997218 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:22.997218 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:22.997218 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:22.997218 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:22.997218 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:22.997218 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:22.997218 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:22.997218 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:22.997218 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 09:46:22.997218 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 09:46:22.997218 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 09:46:22.997218 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 09:46:22.998157 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 09:46:22.998157 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 09:46:22.998157 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 09:46:22.998157 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 09:46:22.998157 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:22.998157 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:22.998157 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:22.998157 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:22.998157 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:22.998157 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:22.998157 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:22.998157 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:22.998157 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:22.998157 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:22.998157 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:22.998157 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:22.998979 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:22.998979 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:22.998979 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:22.998979 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:22.998979 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 09:46:22.998979 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 09:46:22.998979 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 09:46:22.998979 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 09:46:22.998979 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 09:46:22.998979 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 09:46:22.998979 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 09:46:22.998979 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:46:22.998979 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 244/479] 10.222.0.64 () {56 vars in 1114 bytes} [Tue Aug 2 09:46:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:46:23.006275 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b227ef-ee11-4f3e-ab69-5f14b0be8b5e None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:23.124617 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b227ef-ee11-4f3e-ab69-5f14b0be8b5e None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:23.191051 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce0bd5c-635f-4189-9640-c44a6d8b4b1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:23.192011 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce0bd5c-635f-4189-9640-c44a6d8b4b1c None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:23.192478 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce0bd5c-635f-4189-9640-c44a6d8b4b1c None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:23.192896 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce0bd5c-635f-4189-9640-c44a6d8b4b1c None admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:23.194023 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce0bd5c-635f-4189-9640-c44a6d8b4b1c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:23.194515 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce0bd5c-635f-4189-9640-c44a6d8b4b1c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:23.196978 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce0bd5c-635f-4189-9640-c44a6d8b4b1c None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:23.212175 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 236/480] 10.222.0.64 () {56 vars in 1111 bytes} [Tue Aug 2 09:46:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:24.823666 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58e4ff7-08c0-448b-a55d-88798a22d0b8 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:24.824349 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58e4ff7-08c0-448b-a55d-88798a22d0b8 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:24.824349 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58e4ff7-08c0-448b-a55d-88798a22d0b8 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:24.825830 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 245/481] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:46:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:46:24.831489 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d075267-e2db-4a9a-a860-c80c110265e4 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:24.831994 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d075267-e2db-4a9a-a860-c80c110265e4 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:24.832146 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d075267-e2db-4a9a-a860-c80c110265e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:24.862450 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-0d075267-e2db-4a9a-a860-c80c110265e4 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:46:24.863679 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-0d075267-e2db-4a9a-a860-c80c110265e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:46:24.936533 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:24.979039 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 237/482] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:46:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 09:46:25.007981 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82b082f-97af-412f-819d-da330535ecc3 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:25.008336 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82b082f-97af-412f-819d-da330535ecc3 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:25.008581 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82b082f-97af-412f-819d-da330535ecc3 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:25.010232 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 246/483] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:46:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:46:25.012790 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d075267-e2db-4a9a-a860-c80c110265e4 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:25.015698 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4557ef1c-bcf9-47bd-99e6-ae05b730caec None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:25.015986 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4557ef1c-bcf9-47bd-99e6-ae05b730caec None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:25.016288 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4557ef1c-bcf9-47bd-99e6-ae05b730caec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:25.037814 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-4557ef1c-bcf9-47bd-99e6-ae05b730caec None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:46:25.039176 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-4557ef1c-bcf9-47bd-99e6-ae05b730caec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:46:25.129388 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 247/484] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:46:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 09:46:25.158141 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4557ef1c-bcf9-47bd-99e6-ae05b730caec None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:25.162248 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:25.216046 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d075267-e2db-4a9a-a860-c80c110265e4 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:25.293709 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b84dd44-3165-4188-9207-9ceada76c5a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:25.294749 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b84dd44-3165-4188-9207-9ceada76c5a3 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:25.295233 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b84dd44-3165-4188-9207-9ceada76c5a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:25.295726 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b84dd44-3165-4188-9207-9ceada76c5a3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:25.323030 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b84dd44-3165-4188-9207-9ceada76c5a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 09:46:25.337639 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-1b84dd44-3165-4188-9207-9ceada76c5a3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 09:46:25.337639 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:25.337639 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:25.337639 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:25.337639 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:25.337639 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:25.337639 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:25.337639 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:25.337639 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:25.337639 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:25.337639 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:25.337639 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:25.337639 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 09:46:25.337639 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 09:46:25.337639 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 09:46:25.337639 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 09:46:25.338658 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 09:46:25.338658 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 09:46:25.338658 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 09:46:25.338658 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 09:46:25.338658 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:25.338658 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:25.338658 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:25.338658 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:25.338658 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:25.338658 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:25.338658 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:25.338658 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:25.338658 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:25.338658 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:25.338658 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:25.338658 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:25.339541 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:25.339541 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:25.339541 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:25.339541 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:25.339541 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 09:46:25.339541 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 09:46:25.339541 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 09:46:25.339541 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 09:46:25.339541 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 09:46:25.339541 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 09:46:25.339541 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 09:46:25.339541 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:46:25.347155 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 238/485] 10.222.0.64 () {56 vars in 1102 bytes} [Tue Aug 2 09:46:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:46:25.348509 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b84dd44-3165-4188-9207-9ceada76c5a3 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:25.378247 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4557ef1c-bcf9-47bd-99e6-ae05b730caec None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:25.426479 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:25.469009 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bec5be-88bc-45ac-88d5-fbd793cba0a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:25.470158 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bec5be-88bc-45ac-88d5-fbd793cba0a9 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:25.470754 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bec5be-88bc-45ac-88d5-fbd793cba0a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:25.471230 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bec5be-88bc-45ac-88d5-fbd793cba0a9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:25.498304 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bec5be-88bc-45ac-88d5-fbd793cba0a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 09:46:25.508800 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-38bec5be-88bc-45ac-88d5-fbd793cba0a9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 09:46:25.508800 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:25.508800 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:25.508800 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:25.508800 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:25.508800 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:25.508800 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:25.508800 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:25.508800 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:25.508800 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:25.508800 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:25.508800 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:25.508800 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 09:46:25.508800 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 09:46:25.508800 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 09:46:25.508800 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 09:46:25.509769 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 09:46:25.509769 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 09:46:25.509769 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 09:46:25.509769 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 09:46:25.509769 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:25.509769 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:25.509769 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:25.509769 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:25.509769 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:25.509769 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:25.509769 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:25.509769 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:25.509769 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:25.509769 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:25.509769 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:25.509769 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:25.510611 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:25.510611 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:25.510611 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:25.510611 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:25.510611 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 09:46:25.510611 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 09:46:25.510611 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 09:46:25.510611 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 09:46:25.510611 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 09:46:25.510611 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 09:46:25.510611 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 09:46:25.510611 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:46:25.516622 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 248/486] 10.222.0.64 () {56 vars in 1102 bytes} [Tue Aug 2 09:46:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 359 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:46:25.521504 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80edff26-ced1-459e-b1cd-09a890d04c68 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:25.522110 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80edff26-ced1-459e-b1cd-09a890d04c68 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:25.522592 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80edff26-ced1-459e-b1cd-09a890d04c68 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:25.526573 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 249/487] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:46:25 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:46:25.527735 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80edff26-ced1-459e-b1cd-09a890d04c68 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:25.563076 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b84dd44-3165-4188-9207-9ceada76c5a3 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:25.614676 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bb0c78-6295-4eab-94f1-38a590c75bb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:25.617966 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bb0c78-6295-4eab-94f1-38a590c75bb6 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:25.618487 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bb0c78-6295-4eab-94f1-38a590c75bb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:25.618951 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bb0c78-6295-4eab-94f1-38a590c75bb6 None admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:25.620195 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bb0c78-6295-4eab-94f1-38a590c75bb6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:25.625310 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bb0c78-6295-4eab-94f1-38a590c75bb6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:25.633671 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bb0c78-6295-4eab-94f1-38a590c75bb6 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:25.650943 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 239/488] 10.222.0.64 () {56 vars in 1103 bytes} [Tue Aug 2 09:46:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:25.652634 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e07e8f2-bf7b-4167-8d82-fcdf7a370dbf None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:25.653268 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e07e8f2-bf7b-4167-8d82-fcdf7a370dbf None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:25.653745 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e07e8f2-bf7b-4167-8d82-fcdf7a370dbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:25.671538 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:25.689380 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-3e07e8f2-bf7b-4167-8d82-fcdf7a370dbf None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:46:25.691697 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-3e07e8f2-bf7b-4167-8d82-fcdf7a370dbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:46:25.724120 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80edff26-ced1-459e-b1cd-09a890d04c68 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:25.782686 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07afe92-33b3-4d1b-b54b-4da8a209d5dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:25.783303 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07afe92-33b3-4d1b-b54b-4da8a209d5dd None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:25.784276 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07afe92-33b3-4d1b-b54b-4da8a209d5dd None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:25.784768 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07afe92-33b3-4d1b-b54b-4da8a209d5dd None admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:25.786172 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07afe92-33b3-4d1b-b54b-4da8a209d5dd None admin] RBAC: Adding query filter params (name=service) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:25.786491 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07afe92-33b3-4d1b-b54b-4da8a209d5dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:25.787913 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07afe92-33b3-4d1b-b54b-4da8a209d5dd None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:25.797276 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 250/489] 10.222.0.64 () {56 vars in 1103 bytes} [Tue Aug 2 09:46:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:25.803308 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07afe92-33b3-4d1b-b54b-4da8a209d5dd None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:25.820436 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 240/490] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:46:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 09:46:25.820436 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fc31a1-0ef1-49ac-95ce-4b88ef54921b None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:25.820436 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fc31a1-0ef1-49ac-95ce-4b88ef54921b None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:25.820436 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fc31a1-0ef1-49ac-95ce-4b88ef54921b None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:25.820436 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 241/491] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:46:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:46:25.820436 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fea943e-0602-4834-8084-4386ee42f78e None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:25.821174 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fea943e-0602-4834-8084-4386ee42f78e None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:25.821174 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fea943e-0602-4834-8084-4386ee42f78e None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:25.828923 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 242/492] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:46:25 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:46:25.828923 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fea943e-0602-4834-8084-4386ee42f78e None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:25.866534 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:25.926326 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07afe92-33b3-4d1b-b54b-4da8a209d5dd None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:25.950415 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:25.959749 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fea943e-0602-4834-8084-4386ee42f78e None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:25.961442 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707316e2-dee9-461f-869d-3f7e0468c532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:25.962091 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707316e2-dee9-461f-869d-3f7e0468c532 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:25.962380 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707316e2-dee9-461f-869d-3f7e0468c532 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:25.962658 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707316e2-dee9-461f-869d-3f7e0468c532 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:25.974050 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707316e2-dee9-461f-869d-3f7e0468c532 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:25.975778 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707316e2-dee9-461f-869d-3f7e0468c532 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:25.990982 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-707316e2-dee9-461f-869d-3f7e0468c532 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:46:25.990982 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:25.990982 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:25.990982 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:25.990982 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:25.990982 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:25.990982 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:25.990982 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:25.990982 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:25.990982 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:25.990982 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:25.990982 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:25.990982 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 09:46:25.990982 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 09:46:25.990982 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 09:46:25.991900 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 09:46:25.991900 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:25.991900 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:25.991900 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:25.991900 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:25.991900 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:25.991900 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:25.991900 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:25.991900 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:25.991900 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:25.991900 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:25.991900 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:25.991900 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:25.991900 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:25.991900 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:25.991900 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:25.991900 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:25.997589 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 09:46:25.997589 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 09:46:25.997589 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:46:25.997589 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:46:25.997589 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 251/493] 10.222.0.64 () {56 vars in 1108 bytes} [Tue Aug 2 09:46:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:46:25.997589 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6631a2f9-f03d-4aa7-8516-c44104d65c91 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:25.997589 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6631a2f9-f03d-4aa7-8516-c44104d65c91 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:25.997589 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6631a2f9-f03d-4aa7-8516-c44104d65c91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:25.998036 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24e9449-eece-4ed3-9829-9d3fb4d3be78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:25.998036 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24e9449-eece-4ed3-9829-9d3fb4d3be78 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:25.998036 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24e9449-eece-4ed3-9829-9d3fb4d3be78 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:25.998036 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24e9449-eece-4ed3-9829-9d3fb4d3be78 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:26.008572 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24e9449-eece-4ed3-9829-9d3fb4d3be78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:26.010041 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24e9449-eece-4ed3-9829-9d3fb4d3be78 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:26.011903 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-6631a2f9-f03d-4aa7-8516-c44104d65c91 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:46:26.013150 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-6631a2f9-f03d-4aa7-8516-c44104d65c91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:46:26.025646 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-e24e9449-eece-4ed3-9829-9d3fb4d3be78 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:46:26.025646 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:26.025646 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:26.025646 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:26.025646 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:26.025646 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:26.025646 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:26.025646 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:26.025646 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:26.025646 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:26.025646 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:26.025646 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:26.025646 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 09:46:26.025646 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 09:46:26.025646 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 09:46:26.026608 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 09:46:26.026608 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:26.026608 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:26.026608 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:26.026608 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:26.026608 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:26.026608 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:26.026608 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:26.026608 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:26.026608 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:26.026608 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:26.026608 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:26.026608 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:26.026608 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:26.026608 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:26.026608 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:26.026608 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:26.027834 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 09:46:26.027834 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 09:46:26.027834 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:46:26.027834 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:46:26.027834 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 243/494] 10.222.0.64 () {56 vars in 1108 bytes} [Tue Aug 2 09:46:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:46:26.028748 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e39a82-075d-4cd4-9875-e7d29138beee None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:26.028748 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e39a82-075d-4cd4-9875-e7d29138beee None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:26.028748 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e39a82-075d-4cd4-9875-e7d29138beee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:26.049755 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-38e39a82-075d-4cd4-9875-e7d29138beee None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:46:26.051154 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-38e39a82-075d-4cd4-9875-e7d29138beee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:46:26.089126 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 252/495] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:46:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 09:46:26.089982 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6631a2f9-f03d-4aa7-8516-c44104d65c91 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:26.133011 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 244/496] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:46:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 09:46:26.134155 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e39a82-075d-4cd4-9875-e7d29138beee None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:26.151656 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:26.215561 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6631a2f9-f03d-4aa7-8516-c44104d65c91 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:26.223544 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:26.243677 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e39a82-075d-4cd4-9875-e7d29138beee None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:26.259312 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb990a03-24fa-427d-a83d-0e3ba756325e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:26.259312 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb990a03-24fa-427d-a83d-0e3ba756325e None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:26.259312 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb990a03-24fa-427d-a83d-0e3ba756325e None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:26.259312 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb990a03-24fa-427d-a83d-0e3ba756325e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:26.267261 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb990a03-24fa-427d-a83d-0e3ba756325e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 09:46:26.272145 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-eb990a03-24fa-427d-a83d-0e3ba756325e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 09:46:26.272145 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:26.272145 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:26.272145 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:26.272145 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:26.272145 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:26.272145 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:26.272145 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:26.272145 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:26.272145 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:26.272145 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:26.272145 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:26.272145 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 09:46:26.272145 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 09:46:26.272145 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 09:46:26.272145 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 09:46:26.273293 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 09:46:26.273293 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 09:46:26.273293 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 09:46:26.273293 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 09:46:26.273293 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:26.273293 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:26.273293 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:26.273293 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:26.273293 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:26.273293 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:26.273293 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:26.273293 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:26.273293 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:26.273293 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:26.273293 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:26.273293 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:26.274154 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:26.274154 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:26.274154 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:26.274154 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:26.274154 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 09:46:26.274154 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 09:46:26.274154 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 09:46:26.274154 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 09:46:26.274154 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 09:46:26.274154 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 09:46:26.274154 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 09:46:26.274154 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:46:26.274154 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 253/497] 10.222.0.64 () {56 vars in 1102 bytes} [Tue Aug 2 09:46:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:46:26.274154 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb990a03-24fa-427d-a83d-0e3ba756325e None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:26.282876 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4d7b0f-356d-434a-8413-bae2d5471178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:26.283508 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4d7b0f-356d-434a-8413-bae2d5471178 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:26.283795 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4d7b0f-356d-434a-8413-bae2d5471178 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:26.284081 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4d7b0f-356d-434a-8413-bae2d5471178 None admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:26.286285 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4d7b0f-356d-434a-8413-bae2d5471178 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:26.286285 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4d7b0f-356d-434a-8413-bae2d5471178 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:26.287005 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4d7b0f-356d-434a-8413-bae2d5471178 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:26.298707 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 245/498] 10.222.0.64 () {56 vars in 1109 bytes} [Tue Aug 2 09:46:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:26.299567 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4d7b0f-356d-434a-8413-bae2d5471178 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:26.378331 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:26.381962 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb990a03-24fa-427d-a83d-0e3ba756325e None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:26.412635 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4d7b0f-356d-434a-8413-bae2d5471178 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:26.418089 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a01e545-c5d0-4365-a4dd-e57c0dcfbdc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:26.418715 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a01e545-c5d0-4365-a4dd-e57c0dcfbdc5 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:26.419002 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a01e545-c5d0-4365-a4dd-e57c0dcfbdc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:26.419281 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a01e545-c5d0-4365-a4dd-e57c0dcfbdc5 None admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:26.419992 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a01e545-c5d0-4365-a4dd-e57c0dcfbdc5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:26.420336 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a01e545-c5d0-4365-a4dd-e57c0dcfbdc5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:26.421719 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a01e545-c5d0-4365-a4dd-e57c0dcfbdc5 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:26.433742 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 254/499] 10.222.0.64 () {56 vars in 1109 bytes} [Tue Aug 2 09:46:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:26.434608 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a01e545-c5d0-4365-a4dd-e57c0dcfbdc5 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:26.450405 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdecdd4-2990-4043-a5a5-382c570b598b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:26.451053 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdecdd4-2990-4043-a5a5-382c570b598b None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:26.451342 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdecdd4-2990-4043-a5a5-382c570b598b None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:26.451633 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdecdd4-2990-4043-a5a5-382c570b598b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:26.463260 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdecdd4-2990-4043-a5a5-382c570b598b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:26.464788 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdecdd4-2990-4043-a5a5-382c570b598b None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:26.478765 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:26.480813 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-5cdecdd4-2990-4043-a5a5-382c570b598b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:46:26.480813 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:26.480813 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:26.480813 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:26.480813 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:26.480813 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:26.480813 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:26.480813 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:26.480813 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:26.480813 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:26.480813 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:26.480813 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:26.480813 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 09:46:26.480813 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 09:46:26.480813 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 09:46:26.487107 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 09:46:26.487107 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:26.487107 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:26.487107 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:26.487107 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:26.487107 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:26.487107 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:26.487107 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:26.487107 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:26.487107 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:26.487107 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:26.487107 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:26.487107 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:26.487107 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:26.487107 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:26.487107 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:26.487107 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:26.488210 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 09:46:26.488210 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 09:46:26.488210 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:46:26.488210 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:46:26.488210 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 246/500] 10.222.0.64 () {56 vars in 1108 bytes} [Tue Aug 2 09:46:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:46:26.488210 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdecdd4-2990-4043-a5a5-382c570b598b None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:26.563672 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a01e545-c5d0-4365-a4dd-e57c0dcfbdc5 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:26.586149 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdecdd4-2990-4043-a5a5-382c570b598b None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:26.598226 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23811d9-d715-44e6-9051-472a0279ef81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:26.598880 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23811d9-d715-44e6-9051-472a0279ef81 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:26.599171 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23811d9-d715-44e6-9051-472a0279ef81 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:26.599449 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23811d9-d715-44e6-9051-472a0279ef81 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:26.610661 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23811d9-d715-44e6-9051-472a0279ef81 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 09:46:26.616175 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-b23811d9-d715-44e6-9051-472a0279ef81 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 09:46:26.616175 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:26.616175 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:26.616175 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:26.616175 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:26.616175 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:26.616175 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:26.616175 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:26.616175 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:26.616175 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:26.616175 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:26.616175 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:26.616175 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 09:46:26.616175 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 09:46:26.616175 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 09:46:26.616175 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 09:46:26.617384 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 09:46:26.617384 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 09:46:26.617384 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 09:46:26.617384 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 09:46:26.617384 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:26.617384 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:26.617384 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:26.617384 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:26.617384 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:26.617384 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:26.617384 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:26.617384 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:26.617384 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:26.617384 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:26.617384 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:26.617384 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:26.618309 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:26.618309 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:26.618309 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:26.618309 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:26.618309 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 09:46:26.618309 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 09:46:26.618309 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 09:46:26.618309 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 09:46:26.618309 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 09:46:26.618309 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 09:46:26.618309 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 09:46:26.618309 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:46:26.618309 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 255/501] 10.222.0.64 () {56 vars in 1102 bytes} [Tue Aug 2 09:46:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:46:26.618309 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23811d9-d715-44e6-9051-472a0279ef81 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:26.623195 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:26.635638 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8968778a-ad2c-4282-aa36-818ae8f9d37d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:26.636297 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8968778a-ad2c-4282-aa36-818ae8f9d37d None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:26.636584 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8968778a-ad2c-4282-aa36-818ae8f9d37d None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:26.636863 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8968778a-ad2c-4282-aa36-818ae8f9d37d None admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:26.637597 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8968778a-ad2c-4282-aa36-818ae8f9d37d None admin] RBAC: Adding query filter params (name=service) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:26.637909 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8968778a-ad2c-4282-aa36-818ae8f9d37d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:26.639302 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8968778a-ad2c-4282-aa36-818ae8f9d37d None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:26.648621 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 247/502] 10.222.0.64 () {56 vars in 1103 bytes} [Tue Aug 2 09:46:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:26.649454 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8968778a-ad2c-4282-aa36-818ae8f9d37d None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:26.722722 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:26.735897 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23811d9-d715-44e6-9051-472a0279ef81 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:26.750196 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8968778a-ad2c-4282-aa36-818ae8f9d37d None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:26.774141 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23625667-9515-49da-abfd-8727a285a2a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:26.774784 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23625667-9515-49da-abfd-8727a285a2a5 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:26.775067 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23625667-9515-49da-abfd-8727a285a2a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:26.775339 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23625667-9515-49da-abfd-8727a285a2a5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:26.783158 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23625667-9515-49da-abfd-8727a285a2a5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:26.784644 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23625667-9515-49da-abfd-8727a285a2a5 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:26.787461 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7769e7b2-15ef-431b-a3fd-1b2ee863fc4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:26.788114 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7769e7b2-15ef-431b-a3fd-1b2ee863fc4d None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:26.788428 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7769e7b2-15ef-431b-a3fd-1b2ee863fc4d None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:26.788706 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7769e7b2-15ef-431b-a3fd-1b2ee863fc4d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:26.796574 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7769e7b2-15ef-431b-a3fd-1b2ee863fc4d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:26.797628 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-23625667-9515-49da-abfd-8727a285a2a5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 02 09:46:26.797628 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:26.797628 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:26.797628 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:26.797628 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:26.797628 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:26.797628 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:26.797628 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:26.797628 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:26.797628 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:26.797628 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:26.797628 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:26.797628 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 09:46:26.797628 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 09:46:26.797628 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 09:46:26.797628 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 09:46:26.798649 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:26.798649 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:26.798649 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 09:46:26.798649 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 09:46:26.798649 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 09:46:26.798649 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 09:46:26.798649 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:26.798649 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:26.798649 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:26.798649 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:26.798649 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:26.798649 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:26.798649 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:26.798649 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:26.798649 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:26.798649 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:26.798649 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:26.798649 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:26.799573 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:26.799573 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:26.799573 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 09:46:26.799573 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 09:46:26.799573 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 09:46:26.799573 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 09:46:26.799573 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 09:46:26.799573 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 09:46:26.799573 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 02 09:46:26.799573 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:46:26.800097 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7769e7b2-15ef-431b-a3fd-1b2ee863fc4d None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:26.800165 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 256/503] 10.222.0.64 () {56 vars in 1093 bytes} [Tue Aug 2 09:46:26 2022] GET /identity/v3/users/nova => generated 82 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:46:26.800165 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23625667-9515-49da-abfd-8727a285a2a5 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:26.816038 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-7769e7b2-15ef-431b-a3fd-1b2ee863fc4d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 02 09:46:26.816038 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:26.816038 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:26.816038 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:26.816038 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:26.816038 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:26.816038 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:26.816038 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:26.816038 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:26.816038 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:26.816038 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:26.816038 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:26.816038 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 09:46:26.816038 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 09:46:26.816038 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 09:46:26.816038 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 09:46:26.817523 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:26.817523 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:26.817523 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 09:46:26.817523 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 09:46:26.817523 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 09:46:26.817523 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 09:46:26.817523 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:26.817523 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:26.817523 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:26.817523 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:26.817523 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:26.817523 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:26.817523 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:26.817523 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:26.817523 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:26.817523 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:26.817523 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:26.817523 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:26.818820 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:26.818820 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:26.818820 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 09:46:26.818820 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 09:46:26.818820 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 09:46:26.818820 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 09:46:26.818820 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 09:46:26.818820 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 09:46:26.818820 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 02 09:46:26.818820 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:46:26.820096 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 248/504] 10.222.0.64 () {56 vars in 1099 bytes} [Tue Aug 2 09:46:26 2022] GET /identity/v3/users/cinder => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:46:26.821196 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7769e7b2-15ef-431b-a3fd-1b2ee863fc4d None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:26.875717 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:26.914142 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23625667-9515-49da-abfd-8727a285a2a5 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:26.936545 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7769e7b2-15ef-431b-a3fd-1b2ee863fc4d None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:26.958064 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a709735b-3a9f-4ee6-a0e5-913a1976d2bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:26.959007 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a709735b-3a9f-4ee6-a0e5-913a1976d2bd None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:26.959389 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a709735b-3a9f-4ee6-a0e5-913a1976d2bd None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:26.959791 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a709735b-3a9f-4ee6-a0e5-913a1976d2bd None admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:26.960691 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a709735b-3a9f-4ee6-a0e5-913a1976d2bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:26.961093 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a709735b-3a9f-4ee6-a0e5-913a1976d2bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:26.962622 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a709735b-3a9f-4ee6-a0e5-913a1976d2bd None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:26.976798 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 257/505] 10.222.0.64 () {56 vars in 1109 bytes} [Tue Aug 2 09:46:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:26.978018 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a709735b-3a9f-4ee6-a0e5-913a1976d2bd None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:26.978163 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ef1bea-7418-43c4-a765-bcaa97d7a058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:26.978848 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ef1bea-7418-43c4-a765-bcaa97d7a058 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:26.979200 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ef1bea-7418-43c4-a765-bcaa97d7a058 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:26.979540 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ef1bea-7418-43c4-a765-bcaa97d7a058 None admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:26.980487 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ef1bea-7418-43c4-a765-bcaa97d7a058 None admin] RBAC: Adding query filter params (name=service) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:26.981024 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ef1bea-7418-43c4-a765-bcaa97d7a058 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:26.983340 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ef1bea-7418-43c4-a765-bcaa97d7a058 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:26.987556 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:26.995811 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 249/506] 10.222.0.64 () {56 vars in 1103 bytes} [Tue Aug 2 09:46:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:26.996704 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ef1bea-7418-43c4-a765-bcaa97d7a058 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:27.085612 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a709735b-3a9f-4ee6-a0e5-913a1976d2bd None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:27.101078 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ef1bea-7418-43c4-a765-bcaa97d7a058 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:27.108347 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:27.133882 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b34f0c5-b6b7-4a00-8246-7ebeb24538dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:27.134662 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b34f0c5-b6b7-4a00-8246-7ebeb24538dd None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:27.134922 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b34f0c5-b6b7-4a00-8246-7ebeb24538dd None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:27.135178 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b34f0c5-b6b7-4a00-8246-7ebeb24538dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:27.137862 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9062cc86-6052-4de6-934a-f09cf217e62f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:27.138492 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9062cc86-6052-4de6-934a-f09cf217e62f None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:27.138775 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9062cc86-6052-4de6-934a-f09cf217e62f None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:27.139048 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9062cc86-6052-4de6-934a-f09cf217e62f None admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:27.139929 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9062cc86-6052-4de6-934a-f09cf217e62f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:27.140270 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9062cc86-6052-4de6-934a-f09cf217e62f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:27.141889 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9062cc86-6052-4de6-934a-f09cf217e62f None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:27.146784 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b34f0c5-b6b7-4a00-8246-7ebeb24538dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:27.148310 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b34f0c5-b6b7-4a00-8246-7ebeb24538dd None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:27.164791 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-2b34f0c5-b6b7-4a00-8246-7ebeb24538dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:46:27.164791 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:27.164791 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:27.164791 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:27.164791 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:27.164791 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:27.164791 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:27.164791 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:27.164791 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:27.164791 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:27.164791 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:27.164791 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:27.164791 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 09:46:27.164791 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 09:46:27.164791 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 09:46:27.165752 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 09:46:27.165752 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:27.165752 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:27.165752 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:27.165752 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:27.165752 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:27.165752 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:27.165752 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:27.165752 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:27.165752 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:27.165752 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:27.165752 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:27.165752 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:27.165752 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:27.165752 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:27.165752 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:27.165752 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:27.166610 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 09:46:27.166610 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 09:46:27.166610 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:46:27.166610 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:46:27.166610 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 258/507] 10.222.0.64 () {56 vars in 1108 bytes} [Tue Aug 2 09:46:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:46:27.166891 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b34f0c5-b6b7-4a00-8246-7ebeb24538dd None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:27.181675 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 250/508] 10.222.0.64 () {56 vars in 1133 bytes} [Tue Aug 2 09:46:26 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:27.184218 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9062cc86-6052-4de6-934a-f09cf217e62f None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:27.242955 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:27.296046 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9062cc86-6052-4de6-934a-f09cf217e62f None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:27.306572 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b34f0c5-b6b7-4a00-8246-7ebeb24538dd None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:27.335233 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f64fdb8-d0ea-477f-a076-d3cc1717bc0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:27.335865 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f64fdb8-d0ea-477f-a076-d3cc1717bc0b None admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:27.336152 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f64fdb8-d0ea-477f-a076-d3cc1717bc0b None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:27.336884 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f64fdb8-d0ea-477f-a076-d3cc1717bc0b None admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:27.337588 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f64fdb8-d0ea-477f-a076-d3cc1717bc0b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:27.339063 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f64fdb8-d0ea-477f-a076-d3cc1717bc0b None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:27.342204 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963739bc-4eb8-47fe-b661-14ac43f3a080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:27.342826 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963739bc-4eb8-47fe-b661-14ac43f3a080 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:27.343110 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963739bc-4eb8-47fe-b661-14ac43f3a080 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:27.343385 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963739bc-4eb8-47fe-b661-14ac43f3a080 None admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:27.344187 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963739bc-4eb8-47fe-b661-14ac43f3a080 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:27.344524 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963739bc-4eb8-47fe-b661-14ac43f3a080 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:27.346522 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963739bc-4eb8-47fe-b661-14ac43f3a080 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:27.362939 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 251/509] 10.222.0.64 () {60 vars in 1132 bytes} [Tue Aug 2 09:46:27 2022] POST /identity/v3/users => generated 247 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:46:27.363939 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f64fdb8-d0ea-477f-a076-d3cc1717bc0b None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:27.393411 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 259/510] 10.222.0.64 () {56 vars in 1137 bytes} [Tue Aug 2 09:46:27 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:27.393411 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963739bc-4eb8-47fe-b661-14ac43f3a080 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:27.394061 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:27.520628 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963739bc-4eb8-47fe-b661-14ac43f3a080 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:27.530047 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f64fdb8-d0ea-477f-a076-d3cc1717bc0b None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:27.547575 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:27.574632 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9ec83b-49d1-4136-9f34-6d93f6e10225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:27.575296 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9ec83b-49d1-4136-9f34-6d93f6e10225 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:27.575584 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9ec83b-49d1-4136-9f34-6d93f6e10225 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:27.575866 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9ec83b-49d1-4136-9f34-6d93f6e10225 None admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:27.576618 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9ec83b-49d1-4136-9f34-6d93f6e10225 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:27.576931 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9ec83b-49d1-4136-9f34-6d93f6e10225 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:27.578337 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9ec83b-49d1-4136-9f34-6d93f6e10225 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:27.584539 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f205635-f8e0-411e-8fed-33d5a76851f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:27.585635 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f205635-f8e0-411e-8fed-33d5a76851f4 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:27.586413 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f205635-f8e0-411e-8fed-33d5a76851f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:27.586823 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f205635-f8e0-411e-8fed-33d5a76851f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:27.592771 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 260/511] 10.222.0.64 () {56 vars in 1109 bytes} [Tue Aug 2 09:46:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:27.593714 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9ec83b-49d1-4136-9f34-6d93f6e10225 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:27.605696 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f205635-f8e0-411e-8fed-33d5a76851f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:27.608238 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f205635-f8e0-411e-8fed-33d5a76851f4 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:27.635143 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-5f205635-f8e0-411e-8fed-33d5a76851f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:46:27.635143 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:27.635143 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:27.635143 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:27.635143 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:27.635143 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:27.635143 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:27.635143 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:27.635143 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:27.635143 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:27.635143 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:27.635143 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:27.635143 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 09:46:27.635143 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 09:46:27.635143 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 09:46:27.636945 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 09:46:27.636945 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:27.636945 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:27.636945 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:27.636945 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:27.636945 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:27.636945 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:27.636945 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:27.636945 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:27.636945 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:27.636945 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:27.636945 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:27.636945 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:27.636945 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:27.636945 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:27.636945 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:27.636945 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:27.638935 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 09:46:27.638935 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 09:46:27.638935 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:46:27.638935 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:46:27.638935 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 252/512] 10.222.0.64 () {56 vars in 1108 bytes} [Tue Aug 2 09:46:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:46:27.638935 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f205635-f8e0-411e-8fed-33d5a76851f4 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:27.721922 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9ec83b-49d1-4136-9f34-6d93f6e10225 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:27.730803 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:27.766433 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdedf1d-3e52-4d09-bf50-cad677449073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:27.767262 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdedf1d-3e52-4d09-bf50-cad677449073 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:27.767782 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdedf1d-3e52-4d09-bf50-cad677449073 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:27.768158 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdedf1d-3e52-4d09-bf50-cad677449073 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:27.778099 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f205635-f8e0-411e-8fed-33d5a76851f4 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:27.832169 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:27.832663 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6d8247-ded8-4051-a31c-d59f2f7b7705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:27.833864 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6d8247-ded8-4051-a31c-d59f2f7b7705 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:27.834071 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6d8247-ded8-4051-a31c-d59f2f7b7705 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:27.834459 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6d8247-ded8-4051-a31c-d59f2f7b7705 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:27.866934 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdedf1d-3e52-4d09-bf50-cad677449073 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:27.869764 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdedf1d-3e52-4d09-bf50-cad677449073 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:27.913718 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6d8247-ded8-4051-a31c-d59f2f7b7705 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:27.915647 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6d8247-ded8-4051-a31c-d59f2f7b7705 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:27.994175 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 261/513] 10.222.0.64 () {58 vars in 1282 bytes} [Tue Aug 2 09:46:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 09:46:27.995071 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdedf1d-3e52-4d09-bf50-cad677449073 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:28.016040 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:28.060140 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 253/514] 10.222.0.64 () {58 vars in 1282 bytes} [Tue Aug 2 09:46:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 09:46:28.060140 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6d8247-ded8-4051-a31c-d59f2f7b7705 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:28.094183 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:28.121406 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdedf1d-3e52-4d09-bf50-cad677449073 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:28.165265 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe666cf4-0901-46da-8963-1de88a544c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:28.166277 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe666cf4-0901-46da-8963-1de88a544c6e None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:28.166798 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe666cf4-0901-46da-8963-1de88a544c6e None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:28.167259 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe666cf4-0901-46da-8963-1de88a544c6e None admin] PATH_INFO: `/v3/users/glance` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:28.179895 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe666cf4-0901-46da-8963-1de88a544c6e None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:28.182395 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe666cf4-0901-46da-8963-1de88a544c6e None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:28.185617 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6d8247-ded8-4051-a31c-d59f2f7b7705 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:28.197608 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-fe666cf4-0901-46da-8963-1de88a544c6e None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 02 09:46:28.197608 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:28.197608 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:28.197608 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:28.197608 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:28.197608 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:28.197608 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:28.197608 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:28.197608 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:28.197608 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:28.197608 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:28.197608 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:28.197608 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 09:46:28.197608 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 09:46:28.197608 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 09:46:28.197608 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 09:46:28.198633 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:28.198633 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:28.198633 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 09:46:28.198633 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 09:46:28.198633 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 09:46:28.198633 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 09:46:28.198633 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:28.198633 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:28.198633 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:28.198633 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:28.198633 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:28.198633 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:28.198633 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:28.198633 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:28.198633 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:28.198633 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:28.198633 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:28.198633 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:28.200277 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:28.200277 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:28.200277 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 09:46:28.200277 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 09:46:28.200277 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 09:46:28.200277 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 09:46:28.200277 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 09:46:28.200277 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 09:46:28.200277 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 02 09:46:28.200277 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:46:28.200277 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 262/515] 10.222.0.64 () {56 vars in 1099 bytes} [Tue Aug 2 09:46:27 2022] GET /identity/v3/users/glance => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:46:28.200277 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe666cf4-0901-46da-8963-1de88a544c6e None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:28.236997 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9f3650-809a-4cb5-9850-3512eaa74761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:28.237630 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9f3650-809a-4cb5-9850-3512eaa74761 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:28.237920 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9f3650-809a-4cb5-9850-3512eaa74761 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:28.238193 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9f3650-809a-4cb5-9850-3512eaa74761 None admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:28.238910 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9f3650-809a-4cb5-9850-3512eaa74761 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:28.239212 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9f3650-809a-4cb5-9850-3512eaa74761 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:28.240642 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9f3650-809a-4cb5-9850-3512eaa74761 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:28.259224 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 254/516] 10.222.0.64 () {56 vars in 1109 bytes} [Tue Aug 2 09:46:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:28.259224 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9f3650-809a-4cb5-9850-3512eaa74761 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:28.300238 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:28.346194 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe666cf4-0901-46da-8963-1de88a544c6e None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:28.386466 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43b2fba-1589-4c22-be66-8959ef3bd036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:28.387455 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43b2fba-1589-4c22-be66-8959ef3bd036 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:28.387907 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43b2fba-1589-4c22-be66-8959ef3bd036 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:28.388388 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43b2fba-1589-4c22-be66-8959ef3bd036 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:28.391903 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:28.405842 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43b2fba-1589-4c22-be66-8959ef3bd036 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:28.409148 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43b2fba-1589-4c22-be66-8959ef3bd036 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:28.416448 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9f3650-809a-4cb5-9850-3512eaa74761 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:28.437377 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-a43b2fba-1589-4c22-be66-8959ef3bd036 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:46:28.437377 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:28.437377 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:28.437377 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:28.437377 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:28.437377 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:28.437377 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:28.437377 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:28.437377 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:28.437377 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:28.437377 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:28.437377 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:28.437377 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 09:46:28.437377 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 09:46:28.437377 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 09:46:28.438301 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 09:46:28.438301 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:28.438301 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:28.438301 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:28.438301 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:28.438301 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:28.438301 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:28.438301 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:28.438301 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:28.438301 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:28.438301 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:28.438301 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:28.438301 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:28.438301 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:28.438301 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:28.438301 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:28.438301 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:28.439157 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 09:46:28.439157 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 09:46:28.439157 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:46:28.439157 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:46:28.440192 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 263/517] 10.222.0.64 () {56 vars in 1108 bytes} [Tue Aug 2 09:46:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:46:28.440944 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43b2fba-1589-4c22-be66-8959ef3bd036 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:28.458757 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8f70e5-6f22-4a47-9a3d-2d2c9f598a1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:28.459403 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8f70e5-6f22-4a47-9a3d-2d2c9f598a1a None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:28.459686 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8f70e5-6f22-4a47-9a3d-2d2c9f598a1a None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:28.459970 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8f70e5-6f22-4a47-9a3d-2d2c9f598a1a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:28.471833 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8f70e5-6f22-4a47-9a3d-2d2c9f598a1a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:28.473331 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8f70e5-6f22-4a47-9a3d-2d2c9f598a1a None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:28.488997 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-be8f70e5-6f22-4a47-9a3d-2d2c9f598a1a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:46:28.488997 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:28.488997 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:28.488997 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:28.488997 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:28.488997 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:28.488997 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:28.488997 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:28.488997 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:28.488997 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:28.488997 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:28.488997 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:28.488997 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 09:46:28.488997 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 09:46:28.488997 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 09:46:28.489917 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 09:46:28.489917 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:28.489917 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:28.489917 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:28.489917 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:28.489917 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:28.489917 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:28.489917 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:28.489917 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:28.489917 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:28.489917 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:28.489917 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:28.489917 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:28.489917 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:28.489917 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:28.489917 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:28.489917 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:28.490778 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 09:46:28.490778 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 09:46:28.490778 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:46:28.490778 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:46:28.490778 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 255/518] 10.222.0.64 () {56 vars in 1108 bytes} [Tue Aug 2 09:46:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:46:28.491054 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8f70e5-6f22-4a47-9a3d-2d2c9f598a1a None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:28.591317 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:28.618807 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8f70e5-6f22-4a47-9a3d-2d2c9f598a1a None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:28.629066 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43b2fba-1589-4c22-be66-8959ef3bd036 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:28.667234 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72534cd-060f-4f15-85b9-b515a54b0778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:28.667885 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72534cd-060f-4f15-85b9-b515a54b0778 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:28.668227 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72534cd-060f-4f15-85b9-b515a54b0778 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:28.668535 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72534cd-060f-4f15-85b9-b515a54b0778 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:28.674234 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a004e28a-6e1a-462e-8bb2-0f15f0b580b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:28.675153 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a004e28a-6e1a-462e-8bb2-0f15f0b580b0 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:28.675579 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a004e28a-6e1a-462e-8bb2-0f15f0b580b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:28.675998 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a004e28a-6e1a-462e-8bb2-0f15f0b580b0 None admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:28.677103 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a004e28a-6e1a-462e-8bb2-0f15f0b580b0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:28.677344 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72534cd-060f-4f15-85b9-b515a54b0778 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:28.677730 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a004e28a-6e1a-462e-8bb2-0f15f0b580b0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:28.678819 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72534cd-060f-4f15-85b9-b515a54b0778 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:28.679544 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a004e28a-6e1a-462e-8bb2-0f15f0b580b0 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:28.692115 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-c72534cd-060f-4f15-85b9-b515a54b0778 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 02 09:46:28.692115 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:28.692115 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:28.692115 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:28.692115 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:28.692115 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:28.692115 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:28.692115 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:28.692115 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:28.692115 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:28.692115 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:28.692115 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:28.692115 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 09:46:28.692115 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 09:46:28.692115 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 09:46:28.692115 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 09:46:28.693288 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:28.693288 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:28.693288 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 09:46:28.693288 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 09:46:28.693288 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 09:46:28.693288 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 09:46:28.693288 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:28.693288 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:28.693288 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:28.693288 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:28.693288 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:28.693288 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:28.693288 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:28.693288 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:28.693288 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:28.693288 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:28.693288 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:28.693288 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:28.694392 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:28.694392 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:28.694392 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 09:46:28.694392 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 09:46:28.694392 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 09:46:28.694392 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 09:46:28.694392 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 09:46:28.694392 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 09:46:28.694392 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 02 09:46:28.694392 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:46:28.697929 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 256/519] 10.222.0.64 () {56 vars in 1102 bytes} [Tue Aug 2 09:46:28 2022] GET /identity/v3/users/neutron => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:46:28.698847 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72534cd-060f-4f15-85b9-b515a54b0778 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:28.702408 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 264/520] 10.222.0.64 () {56 vars in 1137 bytes} [Tue Aug 2 09:46:28 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:28.703490 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a004e28a-6e1a-462e-8bb2-0f15f0b580b0 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:28.759955 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:28.825403 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72534cd-060f-4f15-85b9-b515a54b0778 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:28.835113 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a004e28a-6e1a-462e-8bb2-0f15f0b580b0 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:28.876070 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144f224f-3b87-4b29-bf39-68d8da6dff24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:28.876645 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144f224f-3b87-4b29-bf39-68d8da6dff24 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:28.876969 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144f224f-3b87-4b29-bf39-68d8da6dff24 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:28.877261 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144f224f-3b87-4b29-bf39-68d8da6dff24 None admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:28.877977 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144f224f-3b87-4b29-bf39-68d8da6dff24 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:28.878284 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144f224f-3b87-4b29-bf39-68d8da6dff24 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:28.879672 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144f224f-3b87-4b29-bf39-68d8da6dff24 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:28.886589 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3417dd4a-af2c-4384-852c-6589b15e569e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:28.887217 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3417dd4a-af2c-4384-852c-6589b15e569e None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:28.887500 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3417dd4a-af2c-4384-852c-6589b15e569e None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:28.887773 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3417dd4a-af2c-4384-852c-6589b15e569e None admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:28.888508 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3417dd4a-af2c-4384-852c-6589b15e569e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:28.888804 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3417dd4a-af2c-4384-852c-6589b15e569e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:28.890192 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3417dd4a-af2c-4384-852c-6589b15e569e None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:28.904680 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 257/521] 10.222.0.64 () {56 vars in 1109 bytes} [Tue Aug 2 09:46:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:28.904680 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144f224f-3b87-4b29-bf39-68d8da6dff24 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:28.904944 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 265/522] 10.222.0.64 () {56 vars in 1109 bytes} [Tue Aug 2 09:46:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:28.904944 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3417dd4a-af2c-4384-852c-6589b15e569e None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:28.920481 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:29.034641 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3417dd4a-af2c-4384-852c-6589b15e569e None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:29.043218 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:29.044646 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144f224f-3b87-4b29-bf39-68d8da6dff24 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:29.075241 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e237ee6c-7d51-440e-8881-1b9810b6a1c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:29.075980 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e237ee6c-7d51-440e-8881-1b9810b6a1c9 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:29.076293 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e237ee6c-7d51-440e-8881-1b9810b6a1c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:29.076578 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e237ee6c-7d51-440e-8881-1b9810b6a1c9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:29.093431 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d9ff7f-aa27-4b9c-b35b-5654f03e50fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:29.094058 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d9ff7f-aa27-4b9c-b35b-5654f03e50fe None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:29.094392 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d9ff7f-aa27-4b9c-b35b-5654f03e50fe None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:29.094606 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d9ff7f-aa27-4b9c-b35b-5654f03e50fe None admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:29.095460 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d9ff7f-aa27-4b9c-b35b-5654f03e50fe None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:29.095789 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d9ff7f-aa27-4b9c-b35b-5654f03e50fe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:29.097263 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d9ff7f-aa27-4b9c-b35b-5654f03e50fe None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:29.116126 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 258/523] 10.222.0.64 () {56 vars in 1139 bytes} [Tue Aug 2 09:46:28 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:29.121568 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d9ff7f-aa27-4b9c-b35b-5654f03e50fe None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:29.167531 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e237ee6c-7d51-440e-8881-1b9810b6a1c9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:29.168979 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e237ee6c-7d51-440e-8881-1b9810b6a1c9 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:29.200325 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:29.253721 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d9ff7f-aa27-4b9c-b35b-5654f03e50fe None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:29.292052 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 266/524] 10.222.0.64 () {58 vars in 1282 bytes} [Tue Aug 2 09:46:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 09:46:29.294513 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e237ee6c-7d51-440e-8881-1b9810b6a1c9 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:29.298397 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2d610a-470f-46e4-bfc5-0c412f5f2769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:29.299065 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2d610a-470f-46e4-bfc5-0c412f5f2769 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:29.299352 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2d610a-470f-46e4-bfc5-0c412f5f2769 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:29.299636 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2d610a-470f-46e4-bfc5-0c412f5f2769 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:29.305351 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:29.309282 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2d610a-470f-46e4-bfc5-0c412f5f2769 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:29.310791 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2d610a-470f-46e4-bfc5-0c412f5f2769 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:29.332685 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-ac2d610a-470f-46e4-bfc5-0c412f5f2769 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 09:46:29.332685 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:29.332685 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:29.332685 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:29.332685 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:29.332685 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:29.332685 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:29.332685 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:29.332685 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:29.332685 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:29.332685 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:29.332685 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:29.332685 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 09:46:29.332685 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 09:46:29.332685 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 09:46:29.333789 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 09:46:29.333789 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:29.333789 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:29.333789 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:29.333789 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:29.333789 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:29.333789 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:29.333789 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:29.333789 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:29.333789 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:29.333789 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:29.333789 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:29.333789 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:29.333789 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:29.333789 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:29.333789 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:29.333789 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:29.334819 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 09:46:29.334819 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 09:46:29.334819 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 09:46:29.334819 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 09:46:29.334819 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 09:46:29.334819 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 09:46:29.334819 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 09:46:29.334819 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:46:29.336757 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 259/525] 10.222.0.64 () {56 vars in 1111 bytes} [Tue Aug 2 09:46:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:46:29.337780 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2d610a-470f-46e4-bfc5-0c412f5f2769 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:29.415979 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e237ee6c-7d51-440e-8881-1b9810b6a1c9 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:29.454452 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c407f940-e05f-42b1-b044-323de69121de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:29.455353 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c407f940-e05f-42b1-b044-323de69121de None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:29.455788 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c407f940-e05f-42b1-b044-323de69121de None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:29.456207 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c407f940-e05f-42b1-b044-323de69121de None admin] PATH_INFO: `/v3/projects/service` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:29.460258 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2d610a-470f-46e4-bfc5-0c412f5f2769 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:29.464475 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c407f940-e05f-42b1-b044-323de69121de None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:29.465953 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c407f940-e05f-42b1-b044-323de69121de None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:29.479445 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-c407f940-e05f-42b1-b044-323de69121de None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 09:46:29.479445 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:29.479445 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:29.479445 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:29.479445 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:29.479445 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:29.479445 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:29.479445 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:29.479445 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:29.479445 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:29.479445 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:29.479445 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:29.479445 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 09:46:29.479445 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 09:46:29.479445 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 09:46:29.480429 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 09:46:29.480429 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:29.480429 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:29.480429 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:29.480429 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:29.480429 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:29.480429 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:29.480429 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:29.480429 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:29.480429 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:29.480429 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:29.480429 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:29.480429 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:29.480429 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:29.480429 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:29.480429 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:29.480429 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:29.481287 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 09:46:29.481287 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 09:46:29.481287 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 09:46:29.481287 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 09:46:29.481287 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 09:46:29.481287 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 09:46:29.481287 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 09:46:29.481287 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:46:29.481287 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 267/526] 10.222.0.64 () {56 vars in 1111 bytes} [Tue Aug 2 09:46:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:46:29.481760 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c407f940-e05f-42b1-b044-323de69121de None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:29.487867 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:29.499536 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda82481-4ab3-4759-b63e-11e0b321949d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:29.500157 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda82481-4ab3-4759-b63e-11e0b321949d None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:29.500469 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda82481-4ab3-4759-b63e-11e0b321949d None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:29.500744 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda82481-4ab3-4759-b63e-11e0b321949d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:29.555639 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:29.571526 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda82481-4ab3-4759-b63e-11e0b321949d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:29.573015 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda82481-4ab3-4759-b63e-11e0b321949d None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:29.591543 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c407f940-e05f-42b1-b044-323de69121de None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:29.627764 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8ac2c3-bd87-4c63-81d4-c838cb7780ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:29.628437 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8ac2c3-bd87-4c63-81d4-c838cb7780ae None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:29.628723 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8ac2c3-bd87-4c63-81d4-c838cb7780ae None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:29.628998 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8ac2c3-bd87-4c63-81d4-c838cb7780ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:29.639869 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8ac2c3-bd87-4c63-81d4-c838cb7780ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:29.641381 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8ac2c3-bd87-4c63-81d4-c838cb7780ae None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:29.656035 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-ea8ac2c3-bd87-4c63-81d4-c838cb7780ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:46:29.656035 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:29.656035 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:29.656035 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:29.656035 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:29.656035 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:29.656035 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:29.656035 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:29.656035 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:29.656035 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:29.656035 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:29.656035 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:29.656035 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 09:46:29.656035 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 09:46:29.656035 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 09:46:29.657068 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 09:46:29.657068 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:29.657068 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:29.657068 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:29.657068 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:29.657068 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:29.657068 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:29.657068 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:29.657068 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:29.657068 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:29.657068 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:29.657068 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:29.657068 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:29.657068 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:29.657068 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:29.657068 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:29.657068 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:29.658499 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 09:46:29.658499 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 09:46:29.658499 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:46:29.658499 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:46:29.658499 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 268/527] 10.222.0.64 () {56 vars in 1108 bytes} [Tue Aug 2 09:46:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:46:29.658788 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8ac2c3-bd87-4c63-81d4-c838cb7780ae None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:29.705428 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 260/528] 10.222.0.64 () {58 vars in 1282 bytes} [Tue Aug 2 09:46:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 09:46:29.706728 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b804208d-0ac5-4ae9-9fdd-1fb77ef9634a None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:29.707020 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b804208d-0ac5-4ae9-9fdd-1fb77ef9634a None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:29.707298 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b804208d-0ac5-4ae9-9fdd-1fb77ef9634a None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:29.708583 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 261/529] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:46:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:46:29.709276 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b804208d-0ac5-4ae9-9fdd-1fb77ef9634a None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:29.746993 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:29.777872 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8ac2c3-bd87-4c63-81d4-c838cb7780ae None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:29.797203 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:29.818492 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c3b9ec-cf7a-40f2-836b-c7591ec0c8ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:29.819192 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c3b9ec-cf7a-40f2-836b-c7591ec0c8ec None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:29.819519 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c3b9ec-cf7a-40f2-836b-c7591ec0c8ec None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:29.819808 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c3b9ec-cf7a-40f2-836b-c7591ec0c8ec None admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:29.820737 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c3b9ec-cf7a-40f2-836b-c7591ec0c8ec None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:29.821122 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c3b9ec-cf7a-40f2-836b-c7591ec0c8ec None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:29.822590 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c3b9ec-cf7a-40f2-836b-c7591ec0c8ec None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:29.842127 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 269/530] 10.222.0.64 () {56 vars in 1148 bytes} [Tue Aug 2 09:46:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:29.842642 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c3b9ec-cf7a-40f2-836b-c7591ec0c8ec None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:29.856925 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b804208d-0ac5-4ae9-9fdd-1fb77ef9634a None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:29.909112 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67c09ea-ba77-4981-a5cd-a5180b55853b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:29.910089 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67c09ea-ba77-4981-a5cd-a5180b55853b None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:29.910510 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67c09ea-ba77-4981-a5cd-a5180b55853b None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:29.910930 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67c09ea-ba77-4981-a5cd-a5180b55853b None admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:29.912017 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67c09ea-ba77-4981-a5cd-a5180b55853b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:29.912411 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67c09ea-ba77-4981-a5cd-a5180b55853b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:29.913860 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67c09ea-ba77-4981-a5cd-a5180b55853b None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:29.933143 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 262/531] 10.222.0.64 () {56 vars in 1148 bytes} [Tue Aug 2 09:46:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:29.934011 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67c09ea-ba77-4981-a5cd-a5180b55853b None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:29.957288 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c3b9ec-cf7a-40f2-836b-c7591ec0c8ec None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:29.998541 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708cc85b-fb9f-4568-a26b-24510a78e3ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:29.999149 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708cc85b-fb9f-4568-a26b-24510a78e3ec None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:29.999434 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708cc85b-fb9f-4568-a26b-24510a78e3ec None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:29.999718 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708cc85b-fb9f-4568-a26b-24510a78e3ec None admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:30.000451 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708cc85b-fb9f-4568-a26b-24510a78e3ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:30.000763 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708cc85b-fb9f-4568-a26b-24510a78e3ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:30.002214 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708cc85b-fb9f-4568-a26b-24510a78e3ec None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:30.014026 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 270/532] 10.222.0.64 () {56 vars in 1109 bytes} [Tue Aug 2 09:46:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:30.015308 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add513c5-d71b-423a-943e-44aa5e795036 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:30.015601 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add513c5-d71b-423a-943e-44aa5e795036 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:30.015879 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add513c5-d71b-423a-943e-44aa5e795036 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:30.026668 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:30.035753 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-add513c5-d71b-423a-943e-44aa5e795036 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:46:30.037024 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-add513c5-d71b-423a-943e-44aa5e795036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:46:30.048784 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67c09ea-ba77-4981-a5cd-a5180b55853b None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:30.074939 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:30.084105 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2e10f1-24df-459d-9e9f-df56f5157f6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:30.084785 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2e10f1-24df-459d-9e9f-df56f5157f6b None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:30.085071 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2e10f1-24df-459d-9e9f-df56f5157f6b None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:30.085345 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2e10f1-24df-459d-9e9f-df56f5157f6b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:30.097357 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2e10f1-24df-459d-9e9f-df56f5157f6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:30.099775 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2e10f1-24df-459d-9e9f-df56f5157f6b None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:30.115401 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 271/533] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:46:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 09:46:30.116338 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add513c5-d71b-423a-943e-44aa5e795036 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:30.118595 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-2f2e10f1-24df-459d-9e9f-df56f5157f6b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:46:30.118595 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:30.118595 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:30.118595 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:30.118595 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:30.118595 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:30.118595 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:30.118595 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:30.118595 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:30.118595 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:30.118595 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:30.118595 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:30.118595 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 09:46:30.118595 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 09:46:30.118595 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 09:46:30.119660 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 09:46:30.119660 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:30.119660 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:30.119660 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:30.119660 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:30.119660 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:30.119660 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:30.119660 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:30.119660 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:30.119660 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:30.119660 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:30.119660 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:30.119660 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:30.119660 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:30.119660 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:30.119660 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:30.119660 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:30.120733 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 09:46:30.120733 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 09:46:30.120733 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:46:30.120733 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:46:30.120733 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 263/534] 10.222.0.64 () {56 vars in 1108 bytes} [Tue Aug 2 09:46:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:46:30.120733 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2e10f1-24df-459d-9e9f-df56f5157f6b None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:30.236192 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add513c5-d71b-423a-943e-44aa5e795036 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:30.243268 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2e10f1-24df-459d-9e9f-df56f5157f6b None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:30.273520 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac07779-d773-4842-a75a-cb5360fb226d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:30.274214 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac07779-d773-4842-a75a-cb5360fb226d None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:30.274423 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac07779-d773-4842-a75a-cb5360fb226d None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:30.274701 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac07779-d773-4842-a75a-cb5360fb226d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:30.275503 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac07779-d773-4842-a75a-cb5360fb226d None admin] RBAC: Adding query filter params (role.id=5704af09562a457fa35364e8af9481f1, scope.project.id=390bd7c4f91c4465b0c7ab0f9e436c4d, user.id=99655e1f9ca24472a86cd26b0dbd1fc2) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:30.275814 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac07779-d773-4842-a75a-cb5360fb226d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:30.277297 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac07779-d773-4842-a75a-cb5360fb226d None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:30.277776 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0686c1-12ce-4ac2-8d6c-a19f87a85e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:30.278395 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0686c1-12ce-4ac2-8d6c-a19f87a85e6f None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:30.278699 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0686c1-12ce-4ac2-8d6c-a19f87a85e6f None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:30.278950 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0686c1-12ce-4ac2-8d6c-a19f87a85e6f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:30.279865 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0686c1-12ce-4ac2-8d6c-a19f87a85e6f None admin] RBAC: Adding query filter params (role.id=5704af09562a457fa35364e8af9481f1, scope.project.id=390bd7c4f91c4465b0c7ab0f9e436c4d, user.id=10d7de62b9ee4039bdc79627ac7fb499) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:30.280180 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0686c1-12ce-4ac2-8d6c-a19f87a85e6f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:30.281653 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0686c1-12ce-4ac2-8d6c-a19f87a85e6f None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:30.284017 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 272/535] 10.222.0.64 () {56 vars in 1374 bytes} [Tue Aug 2 09:46:30 2022] GET /identity/v3/role_assignments?user.id=99655e1f9ca24472a86cd26b0dbd1fc2&scope.project.id=390bd7c4f91c4465b0c7ab0f9e436c4d&role.id=5704af09562a457fa35364e8af9481f1 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:30.284876 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac07779-d773-4842-a75a-cb5360fb226d None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:30.288290 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 264/536] 10.222.0.64 () {56 vars in 1374 bytes} [Tue Aug 2 09:46:30 2022] GET /identity/v3/role_assignments?user.id=10d7de62b9ee4039bdc79627ac7fb499&scope.project.id=390bd7c4f91c4465b0c7ab0f9e436c4d&role.id=5704af09562a457fa35364e8af9481f1 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:30.289081 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0686c1-12ce-4ac2-8d6c-a19f87a85e6f None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:30.298014 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:30.368679 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:30.425789 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac07779-d773-4842-a75a-cb5360fb226d None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:30.460832 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0686c1-12ce-4ac2-8d6c-a19f87a85e6f None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:30.471612 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2d8c6c-b3f0-41d3-9724-b4b3c2ad6be6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:30.472294 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2d8c6c-b3f0-41d3-9724-b4b3c2ad6be6 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:30.472582 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2d8c6c-b3f0-41d3-9724-b4b3c2ad6be6 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:30.472856 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2d8c6c-b3f0-41d3-9724-b4b3c2ad6be6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:30.481673 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2d8c6c-b3f0-41d3-9724-b4b3c2ad6be6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:30.483172 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2d8c6c-b3f0-41d3-9724-b4b3c2ad6be6 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:30.495819 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-ce2d8c6c-b3f0-41d3-9724-b4b3c2ad6be6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 09:46:30.495819 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:30.495819 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:30.495819 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:30.495819 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:30.495819 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:30.495819 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:30.495819 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:30.495819 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:30.495819 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:30.495819 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:30.495819 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:30.495819 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 09:46:30.495819 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 09:46:30.495819 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 09:46:30.496749 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 09:46:30.496749 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:30.496749 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:30.496749 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:30.496749 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:30.496749 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:30.496749 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:30.496749 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:30.496749 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:30.496749 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:30.496749 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:30.496749 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:30.496749 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:30.496749 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:30.496749 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:30.496749 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:30.496749 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:30.497610 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 09:46:30.497610 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 09:46:30.497610 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 09:46:30.497610 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 09:46:30.497610 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 09:46:30.497610 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 09:46:30.497610 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 09:46:30.497610 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:46:30.497610 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 273/537] 10.222.0.64 () {56 vars in 1111 bytes} [Tue Aug 2 09:46:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:46:30.498546 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2d8c6c-b3f0-41d3-9724-b4b3c2ad6be6 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:30.517582 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d207961-27d1-42b2-b18e-e1155931e225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:30.518210 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d207961-27d1-42b2-b18e-e1155931e225 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:30.518495 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d207961-27d1-42b2-b18e-e1155931e225 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:30.518767 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d207961-27d1-42b2-b18e-e1155931e225 None admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:30.519473 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d207961-27d1-42b2-b18e-e1155931e225 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:30.519777 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d207961-27d1-42b2-b18e-e1155931e225 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:30.521224 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d207961-27d1-42b2-b18e-e1155931e225 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:30.536624 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 265/538] 10.222.0.64 () {56 vars in 1109 bytes} [Tue Aug 2 09:46:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:30.538213 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d207961-27d1-42b2-b18e-e1155931e225 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:30.610518 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:30.620001 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2d8c6c-b3f0-41d3-9724-b4b3c2ad6be6 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:30.642413 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:30.659743 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ddd4ce-bc1c-49d7-a371-8eb94f721d19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:30.660497 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ddd4ce-bc1c-49d7-a371-8eb94f721d19 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:30.660784 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ddd4ce-bc1c-49d7-a371-8eb94f721d19 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:30.661060 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ddd4ce-bc1c-49d7-a371-8eb94f721d19 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:30.673783 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ddd4ce-bc1c-49d7-a371-8eb94f721d19 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 09:46:30.684366 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-b8ddd4ce-bc1c-49d7-a371-8eb94f721d19 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 09:46:30.684366 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:30.684366 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:30.684366 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:30.684366 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:30.684366 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:30.684366 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:30.684366 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:30.684366 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:30.684366 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:30.684366 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:30.684366 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:30.684366 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 09:46:30.684366 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 09:46:30.684366 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 09:46:30.684366 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 09:46:30.685328 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 09:46:30.685328 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 09:46:30.685328 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 09:46:30.685328 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 09:46:30.685328 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:30.685328 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:30.685328 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:30.685328 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:30.685328 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:30.685328 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:30.685328 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:30.685328 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:30.685328 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:30.685328 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:30.685328 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:30.685328 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:30.686141 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:30.686141 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:30.686141 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:30.686141 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:30.686141 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 09:46:30.686141 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 09:46:30.686141 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 09:46:30.686141 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 09:46:30.686141 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 09:46:30.686141 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 09:46:30.686141 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 09:46:30.686141 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:46:30.686141 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 274/539] 10.222.0.64 () {56 vars in 1102 bytes} [Tue Aug 2 09:46:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:46:30.686141 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ddd4ce-bc1c-49d7-a371-8eb94f721d19 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:30.711315 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d207961-27d1-42b2-b18e-e1155931e225 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:30.751795 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e1d7d7-1c9d-4eb6-a9ad-f068c01a5084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:30.752318 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e1d7d7-1c9d-4eb6-a9ad-f068c01a5084 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:30.752617 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e1d7d7-1c9d-4eb6-a9ad-f068c01a5084 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:30.752900 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e1d7d7-1c9d-4eb6-a9ad-f068c01a5084 None admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:30.753683 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e1d7d7-1c9d-4eb6-a9ad-f068c01a5084 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:30.754003 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e1d7d7-1c9d-4eb6-a9ad-f068c01a5084 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:30.755520 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e1d7d7-1c9d-4eb6-a9ad-f068c01a5084 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:30.774744 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 266/540] 10.222.0.64 () {56 vars in 1148 bytes} [Tue Aug 2 09:46:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:30.775821 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e1d7d7-1c9d-4eb6-a9ad-f068c01a5084 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:30.830484 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ddd4ce-bc1c-49d7-a371-8eb94f721d19 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:30.880568 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e00dc83-5490-4df4-a77e-28217ab55007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:30.881362 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e00dc83-5490-4df4-a77e-28217ab55007 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:30.881716 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e00dc83-5490-4df4-a77e-28217ab55007 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:30.882062 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e00dc83-5490-4df4-a77e-28217ab55007 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:30.893458 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e00dc83-5490-4df4-a77e-28217ab55007 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:30.896464 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e00dc83-5490-4df4-a77e-28217ab55007 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:30.916689 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e1d7d7-1c9d-4eb6-a9ad-f068c01a5084 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:30.919465 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-7e00dc83-5490-4df4-a77e-28217ab55007 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 09:46:30.919465 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:30.919465 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:30.919465 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:30.919465 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:30.919465 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:30.919465 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:30.919465 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:30.919465 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:30.919465 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:30.919465 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:30.919465 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:30.919465 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 09:46:30.919465 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 09:46:30.919465 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 09:46:30.920342 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 09:46:30.920342 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:30.920342 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:30.920342 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:30.920342 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:30.920342 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:30.920342 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:30.920342 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:30.920342 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:30.920342 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:30.920342 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:30.920342 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:30.920342 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:30.920342 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:30.920342 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:30.920342 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:30.920342 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:30.921233 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 09:46:30.921233 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 09:46:30.921233 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 09:46:30.921233 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 09:46:30.921233 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 09:46:30.921233 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 09:46:30.921233 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 09:46:30.921233 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:46:30.923883 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 275/541] 10.222.0.64 () {56 vars in 1111 bytes} [Tue Aug 2 09:46:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:46:30.925126 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e00dc83-5490-4df4-a77e-28217ab55007 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:30.935708 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:30.949149 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:30.958086 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7672be9-9730-43c5-a0e3-4ab385be960f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:30.959512 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7672be9-9730-43c5-a0e3-4ab385be960f None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:30.960327 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7672be9-9730-43c5-a0e3-4ab385be960f None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:30.960838 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7672be9-9730-43c5-a0e3-4ab385be960f None admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:30.961954 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7672be9-9730-43c5-a0e3-4ab385be960f None admin] RBAC: Adding query filter params (name=service) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:30.962775 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7672be9-9730-43c5-a0e3-4ab385be960f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:30.964829 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7672be9-9730-43c5-a0e3-4ab385be960f None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:30.979576 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 267/542] 10.222.0.64 () {56 vars in 1103 bytes} [Tue Aug 2 09:46:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:30.980752 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7672be9-9730-43c5-a0e3-4ab385be960f None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:31.049028 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e00dc83-5490-4df4-a77e-28217ab55007 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:31.087528 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1679f680-d6e2-48f2-9490-ea4916c4f6ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:31.088623 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1679f680-d6e2-48f2-9490-ea4916c4f6ce None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:31.089116 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1679f680-d6e2-48f2-9490-ea4916c4f6ce None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:31.089620 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1679f680-d6e2-48f2-9490-ea4916c4f6ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:31.091022 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1679f680-d6e2-48f2-9490-ea4916c4f6ce None admin] RBAC: Adding query filter params (role.id=5704af09562a457fa35364e8af9481f1, scope.project.id=390bd7c4f91c4465b0c7ab0f9e436c4d, user.id=04fb1dbe260a4721aaa0bd0493260d33) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:31.091602 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1679f680-d6e2-48f2-9490-ea4916c4f6ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:31.093287 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1679f680-d6e2-48f2-9490-ea4916c4f6ce None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:31.100931 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 276/543] 10.222.0.64 () {56 vars in 1374 bytes} [Tue Aug 2 09:46:30 2022] GET /identity/v3/role_assignments?user.id=04fb1dbe260a4721aaa0bd0493260d33&scope.project.id=390bd7c4f91c4465b0c7ab0f9e436c4d&role.id=5704af09562a457fa35364e8af9481f1 => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:31.102191 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1679f680-d6e2-48f2-9490-ea4916c4f6ce None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:31.142795 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7672be9-9730-43c5-a0e3-4ab385be960f None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:31.196640 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:31.230149 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1679f680-d6e2-48f2-9490-ea4916c4f6ce None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:31.273665 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf57398a-331c-4f51-aec1-cbf696b62d82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:31.274007 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28deb2c3-6f98-4e4f-9071-ef3bf02bbe9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:31.274313 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf57398a-331c-4f51-aec1-cbf696b62d82 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:31.274597 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf57398a-331c-4f51-aec1-cbf696b62d82 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:31.274872 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf57398a-331c-4f51-aec1-cbf696b62d82 None admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:31.275046 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28deb2c3-6f98-4e4f-9071-ef3bf02bbe9c None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:31.275433 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28deb2c3-6f98-4e4f-9071-ef3bf02bbe9c None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:31.275650 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf57398a-331c-4f51-aec1-cbf696b62d82 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:31.275906 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28deb2c3-6f98-4e4f-9071-ef3bf02bbe9c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:31.276004 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf57398a-331c-4f51-aec1-cbf696b62d82 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:31.277484 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf57398a-331c-4f51-aec1-cbf696b62d82 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:31.296045 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28deb2c3-6f98-4e4f-9071-ef3bf02bbe9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:31.297755 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28deb2c3-6f98-4e4f-9071-ef3bf02bbe9c None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:31.315888 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 268/544] 10.222.0.64 () {56 vars in 1148 bytes} [Tue Aug 2 09:46:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:31.321765 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf57398a-331c-4f51-aec1-cbf696b62d82 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:31.325912 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-28deb2c3-6f98-4e4f-9071-ef3bf02bbe9c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:46:31.325912 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:31.325912 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:31.325912 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:31.325912 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:31.325912 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:31.325912 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:31.325912 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:31.325912 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:31.325912 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:31.325912 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:31.325912 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:31.325912 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 09:46:31.325912 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 09:46:31.325912 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 09:46:31.326776 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 09:46:31.326776 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:31.326776 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:31.326776 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:31.326776 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:31.326776 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:31.326776 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:31.326776 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:31.326776 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:31.326776 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:31.326776 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:31.326776 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:31.326776 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:31.326776 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:31.326776 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:31.326776 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:31.326776 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:31.327673 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 09:46:31.327673 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 09:46:31.327673 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:46:31.327673 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:46:31.328347 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 277/545] 10.222.0.64 () {56 vars in 1108 bytes} [Tue Aug 2 09:46:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:46:31.333982 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28deb2c3-6f98-4e4f-9071-ef3bf02bbe9c None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:31.448885 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:31.516231 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28deb2c3-6f98-4e4f-9071-ef3bf02bbe9c None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:31.564825 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:31.577811 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4a559e-9434-418d-b118-3796351e7896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:31.578701 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4a559e-9434-418d-b118-3796351e7896 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:31.579682 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4a559e-9434-418d-b118-3796351e7896 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:31.580120 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4a559e-9434-418d-b118-3796351e7896 None admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:31.581377 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4a559e-9434-418d-b118-3796351e7896 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:31.582467 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4a559e-9434-418d-b118-3796351e7896 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:31.586449 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4a559e-9434-418d-b118-3796351e7896 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:31.592553 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf57398a-331c-4f51-aec1-cbf696b62d82 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:31.630485 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 278/546] 10.222.0.64 () {56 vars in 1109 bytes} [Tue Aug 2 09:46:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:46:31.637819 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4a559e-9434-418d-b118-3796351e7896 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:31.677742 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdda90bc-20cb-47a2-aa17-50422231de9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:31.678560 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdda90bc-20cb-47a2-aa17-50422231de9d None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:31.678921 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdda90bc-20cb-47a2-aa17-50422231de9d None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:31.679207 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdda90bc-20cb-47a2-aa17-50422231de9d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:31.680195 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdda90bc-20cb-47a2-aa17-50422231de9d None admin] RBAC: Adding query filter params (role.id=5704af09562a457fa35364e8af9481f1, scope.project.id=390bd7c4f91c4465b0c7ab0f9e436c4d, user.id=ec31b188970947efb2ed1d4d9a31cadf) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:31.680540 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdda90bc-20cb-47a2-aa17-50422231de9d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:31.682297 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdda90bc-20cb-47a2-aa17-50422231de9d None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:31.689006 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 269/547] 10.222.0.64 () {56 vars in 1374 bytes} [Tue Aug 2 09:46:31 2022] GET /identity/v3/role_assignments?user.id=ec31b188970947efb2ed1d4d9a31cadf&scope.project.id=390bd7c4f91c4465b0c7ab0f9e436c4d&role.id=5704af09562a457fa35364e8af9481f1 => generated 259 bytes in 368 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:46:31.829690 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4a559e-9434-418d-b118-3796351e7896 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:31.897485 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18f92dd-8a1a-4dee-9299-970df0859734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:31.898507 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18f92dd-8a1a-4dee-9299-970df0859734 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:31.898943 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18f92dd-8a1a-4dee-9299-970df0859734 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:31.899374 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18f92dd-8a1a-4dee-9299-970df0859734 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:31.921762 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18f92dd-8a1a-4dee-9299-970df0859734 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:31.923782 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18f92dd-8a1a-4dee-9299-970df0859734 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:31.939916 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-c18f92dd-8a1a-4dee-9299-970df0859734 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 02 09:46:31.939916 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:31.939916 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:31.939916 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:31.939916 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:31.939916 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:31.939916 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:31.939916 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:31.939916 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:31.939916 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:31.939916 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:31.939916 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:31.939916 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 09:46:31.939916 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 09:46:31.939916 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 09:46:31.939916 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 09:46:31.940921 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:31.940921 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:31.940921 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 09:46:31.940921 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 09:46:31.940921 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 09:46:31.940921 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 09:46:31.940921 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:31.940921 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:31.940921 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:31.940921 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:31.940921 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:31.940921 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:31.940921 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:31.940921 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:31.940921 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:31.940921 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:31.940921 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:31.940921 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:31.942442 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:31.942442 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:31.942442 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 09:46:31.942442 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 09:46:31.942442 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 09:46:31.942442 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 09:46:31.942442 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 09:46:31.942442 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 09:46:31.942442 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 02 09:46:31.942442 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:46:31.945744 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 279/548] 10.222.0.64 () {56 vars in 1096 bytes} [Tue Aug 2 09:46:31 2022] GET /identity/v3/users/swift => generated 83 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 09:46:31.949414 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdda90bc-20cb-47a2-aa17-50422231de9d None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:32.080514 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdda90bc-20cb-47a2-aa17-50422231de9d None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:32.084144 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:32.148360 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d856dba-b511-4f90-85c9-556cb12b406d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:32.149063 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d856dba-b511-4f90-85c9-556cb12b406d None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:32.149347 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d856dba-b511-4f90-85c9-556cb12b406d None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:32.149623 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d856dba-b511-4f90-85c9-556cb12b406d None admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:32.150501 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d856dba-b511-4f90-85c9-556cb12b406d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:32.150812 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d856dba-b511-4f90-85c9-556cb12b406d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:32.152273 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d856dba-b511-4f90-85c9-556cb12b406d None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:32.181828 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 270/549] 10.222.0.64 () {56 vars in 1135 bytes} [Tue Aug 2 09:46:31 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:32.186727 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18f92dd-8a1a-4dee-9299-970df0859734 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:32.260367 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:32.338562 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18f92dd-8a1a-4dee-9299-970df0859734 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:32.397566 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b35cb9-7299-4621-8746-88a5c7b145fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:32.398200 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b35cb9-7299-4621-8746-88a5c7b145fd None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:32.398483 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b35cb9-7299-4621-8746-88a5c7b145fd None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:32.398756 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b35cb9-7299-4621-8746-88a5c7b145fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:32.489310 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b35cb9-7299-4621-8746-88a5c7b145fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:32.490803 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b35cb9-7299-4621-8746-88a5c7b145fd None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:32.548089 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:32.611002 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 280/550] 10.222.0.64 () {58 vars in 1282 bytes} [Tue Aug 2 09:46:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 09:46:32.619411 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d856dba-b511-4f90-85c9-556cb12b406d None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:32.764375 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c161a0-a20d-4441-b10f-73df50a6d51f None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:32.765028 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c161a0-a20d-4441-b10f-73df50a6d51f None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:32.765539 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c161a0-a20d-4441-b10f-73df50a6d51f None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:32.767638 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 281/551] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:46:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 09:46:32.772671 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d856dba-b511-4f90-85c9-556cb12b406d None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:32.773584 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629f6623-fda6-47da-bc57-26a0ce46230e None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:32.774088 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629f6623-fda6-47da-bc57-26a0ce46230e None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:32.774668 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629f6623-fda6-47da-bc57-26a0ce46230e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:32.812202 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4863c4a0-1a3d-481e-9e6c-2c0ef5151fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:32.812877 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4863c4a0-1a3d-481e-9e6c-2c0ef5151fba None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:32.813160 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4863c4a0-1a3d-481e-9e6c-2c0ef5151fba None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:32.813466 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4863c4a0-1a3d-481e-9e6c-2c0ef5151fba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:32.820532 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-629f6623-fda6-47da-bc57-26a0ce46230e None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:46:32.821743 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-629f6623-fda6-47da-bc57-26a0ce46230e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:46:32.837221 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4863c4a0-1a3d-481e-9e6c-2c0ef5151fba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:32.838695 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4863c4a0-1a3d-481e-9e6c-2c0ef5151fba None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:32.876457 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-4863c4a0-1a3d-481e-9e6c-2c0ef5151fba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:46:32.876457 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:32.876457 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:32.876457 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:32.876457 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:32.876457 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:32.876457 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:32.876457 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:32.876457 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:32.876457 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:32.876457 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:32.876457 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:32.876457 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 09:46:32.876457 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 09:46:32.876457 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 09:46:32.877518 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 09:46:32.877518 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:32.877518 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:32.877518 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:32.877518 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:32.877518 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:32.877518 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:32.877518 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:32.877518 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:32.877518 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:32.877518 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:32.877518 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:32.877518 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:32.877518 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:32.877518 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:32.877518 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:32.877518 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:32.878418 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 09:46:32.878418 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 09:46:32.878418 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:46:32.878418 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:46:32.878418 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 271/552] 10.222.0.64 () {56 vars in 1108 bytes} [Tue Aug 2 09:46:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:46:32.886104 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4863c4a0-1a3d-481e-9e6c-2c0ef5151fba None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:32.914507 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:32.951198 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 282/553] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:46:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 09:46:32.960522 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6263549f-2166-4cd7-a2ea-245db128b0f6 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:32.960821 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6263549f-2166-4cd7-a2ea-245db128b0f6 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:32.961110 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6263549f-2166-4cd7-a2ea-245db128b0f6 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:32.962988 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 283/554] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:46:32 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:46:32.972232 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07b1ded-13e3-4fde-bbd9-ddeb210068a0 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:32.972557 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07b1ded-13e3-4fde-bbd9-ddeb210068a0 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:32.972836 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07b1ded-13e3-4fde-bbd9-ddeb210068a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:32.999722 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-b07b1ded-13e3-4fde-bbd9-ddeb210068a0 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:46:33.001804 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-b07b1ded-13e3-4fde-bbd9-ddeb210068a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:46:33.027777 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4863c4a0-1a3d-481e-9e6c-2c0ef5151fba None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:33.070650 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bda81d6-71df-45fd-819a-34dee339aa03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:33.071543 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bda81d6-71df-45fd-819a-34dee339aa03 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:33.071993 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bda81d6-71df-45fd-819a-34dee339aa03 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:33.072376 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bda81d6-71df-45fd-819a-34dee339aa03 None admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:33.073308 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bda81d6-71df-45fd-819a-34dee339aa03 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:33.073743 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bda81d6-71df-45fd-819a-34dee339aa03 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:33.075242 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bda81d6-71df-45fd-819a-34dee339aa03 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:33.093946 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 284/555] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:46:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 09:46:33.095022 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07b1ded-13e3-4fde-bbd9-ddeb210068a0 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:33.096358 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 272/556] 10.222.0.64 () {56 vars in 1109 bytes} [Tue Aug 2 09:46:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:33.106066 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:33.109485 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bda81d6-71df-45fd-819a-34dee339aa03 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:33.216043 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:33.244370 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bda81d6-71df-45fd-819a-34dee339aa03 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:33.245529 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07b1ded-13e3-4fde-bbd9-ddeb210068a0 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:33.283507 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dfbd90-347a-436c-9249-c3061c0b2b56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:33.284569 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dfbd90-347a-436c-9249-c3061c0b2b56 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:33.285170 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dfbd90-347a-436c-9249-c3061c0b2b56 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:33.285649 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dfbd90-347a-436c-9249-c3061c0b2b56 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:33.289097 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af61fa5-9e8f-42e9-aa67-c1f2f13cadd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:33.289647 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af61fa5-9e8f-42e9-aa67-c1f2f13cadd1 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:33.289938 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af61fa5-9e8f-42e9-aa67-c1f2f13cadd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:33.290217 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af61fa5-9e8f-42e9-aa67-c1f2f13cadd1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:33.294326 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dfbd90-347a-436c-9249-c3061c0b2b56 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:33.296048 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dfbd90-347a-436c-9249-c3061c0b2b56 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:33.306309 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af61fa5-9e8f-42e9-aa67-c1f2f13cadd1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 09:46:33.308793 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-a0dfbd90-347a-436c-9249-c3061c0b2b56 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 09:46:33.308793 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:33.308793 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:33.308793 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:33.308793 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:33.308793 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:33.308793 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:33.308793 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:33.308793 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:33.308793 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:33.308793 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:33.308793 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:33.308793 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 09:46:33.308793 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 09:46:33.308793 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 09:46:33.309653 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 09:46:33.309653 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:33.309653 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:33.309653 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:33.309653 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:33.309653 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:33.309653 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:33.309653 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:33.309653 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:33.309653 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:33.309653 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:33.309653 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:33.309653 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:33.309653 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:33.309653 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:33.309653 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:33.309653 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:33.310570 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 09:46:33.310570 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 09:46:33.310570 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 09:46:33.310570 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 09:46:33.310570 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 09:46:33.310570 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 09:46:33.310570 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 09:46:33.310570 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:46:33.317574 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-2af61fa5-9e8f-42e9-aa67-c1f2f13cadd1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 09:46:33.317574 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:33.317574 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:33.317574 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:33.317574 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:33.317574 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:33.317574 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:33.317574 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:33.317574 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:33.317574 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:33.317574 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:33.317574 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:33.317574 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 09:46:33.317574 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 09:46:33.317574 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 09:46:33.317574 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 09:46:33.318579 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 09:46:33.318579 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 09:46:33.318579 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 09:46:33.318579 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 09:46:33.318579 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:33.318579 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:33.318579 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:33.318579 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:33.318579 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:33.318579 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:33.318579 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:33.318579 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:33.318579 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:33.318579 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:33.318579 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:33.318579 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:33.319959 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:33.319959 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:33.319959 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:33.319959 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:33.319959 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 09:46:33.319959 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 09:46:33.319959 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 09:46:33.319959 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 09:46:33.319959 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 09:46:33.319959 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 09:46:33.319959 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 09:46:33.319959 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:46:33.321557 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 285/558] 10.222.0.64 () {56 vars in 1102 bytes} [Tue Aug 2 09:46:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:46:33.321779 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 273/558] 10.222.0.64 () {56 vars in 1111 bytes} [Tue Aug 2 09:46:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:46:33.322464 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af61fa5-9e8f-42e9-aa67-c1f2f13cadd1 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:33.323254 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dfbd90-347a-436c-9249-c3061c0b2b56 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:33.430229 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:33.431170 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dfbd90-347a-436c-9249-c3061c0b2b56 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:33.469068 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7489063d-de5b-4490-9ba1-21f0af1f6063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:33.470103 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7489063d-de5b-4490-9ba1-21f0af1f6063 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:33.470597 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7489063d-de5b-4490-9ba1-21f0af1f6063 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:33.471080 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7489063d-de5b-4490-9ba1-21f0af1f6063 None admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:33.472023 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7489063d-de5b-4490-9ba1-21f0af1f6063 None admin] RBAC: Adding query filter params (name=service) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:33.472217 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af61fa5-9e8f-42e9-aa67-c1f2f13cadd1 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:33.472743 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7489063d-de5b-4490-9ba1-21f0af1f6063 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:33.474341 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7489063d-de5b-4490-9ba1-21f0af1f6063 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:33.476596 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:33.485314 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 274/559] 10.222.0.64 () {56 vars in 1103 bytes} [Tue Aug 2 09:46:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:33.486667 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7489063d-de5b-4490-9ba1-21f0af1f6063 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:33.522898 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cea9e9-40e2-4f4e-a818-c37abd80dc16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:33.523549 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cea9e9-40e2-4f4e-a818-c37abd80dc16 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:33.523843 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cea9e9-40e2-4f4e-a818-c37abd80dc16 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:33.524116 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cea9e9-40e2-4f4e-a818-c37abd80dc16 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:33.536797 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cea9e9-40e2-4f4e-a818-c37abd80dc16 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 09:46:33.541773 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-49cea9e9-40e2-4f4e-a818-c37abd80dc16 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 09:46:33.541773 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:33.541773 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:33.541773 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:33.541773 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:33.541773 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:33.541773 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:33.541773 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:33.541773 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:33.541773 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:33.541773 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:33.541773 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:33.541773 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 09:46:33.541773 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 09:46:33.541773 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 09:46:33.541773 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 09:46:33.543116 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 09:46:33.543116 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 09:46:33.543116 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 09:46:33.543116 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 09:46:33.543116 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:33.543116 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:33.543116 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:33.543116 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:33.543116 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:33.543116 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:33.543116 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:33.543116 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:33.543116 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:33.543116 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:33.543116 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:33.543116 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:33.544470 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:33.544470 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:33.544470 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:33.544470 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:33.544470 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 09:46:33.544470 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 09:46:33.544470 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 09:46:33.544470 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 09:46:33.544470 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 09:46:33.544470 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 09:46:33.544470 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 09:46:33.544470 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:46:33.547954 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 286/560] 10.222.0.64 () {56 vars in 1102 bytes} [Tue Aug 2 09:46:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:46:33.549411 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cea9e9-40e2-4f4e-a818-c37abd80dc16 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:33.621701 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7489063d-de5b-4490-9ba1-21f0af1f6063 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:33.659608 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f8a6f0-2948-46a7-8668-f868c243ae57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:33.660625 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f8a6f0-2948-46a7-8668-f868c243ae57 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:33.661138 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f8a6f0-2948-46a7-8668-f868c243ae57 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:33.661629 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f8a6f0-2948-46a7-8668-f868c243ae57 None admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:33.662488 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f8a6f0-2948-46a7-8668-f868c243ae57 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:33.662798 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f8a6f0-2948-46a7-8668-f868c243ae57 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:33.664330 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f8a6f0-2948-46a7-8668-f868c243ae57 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:33.677960 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 275/561] 10.222.0.64 () {56 vars in 1148 bytes} [Tue Aug 2 09:46:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:33.678824 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f8a6f0-2948-46a7-8668-f868c243ae57 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:33.687763 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cea9e9-40e2-4f4e-a818-c37abd80dc16 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:33.730522 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:33.744335 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b03c90-c463-48e1-8378-8542999ddad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:33.745663 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b03c90-c463-48e1-8378-8542999ddad1 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:33.746121 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b03c90-c463-48e1-8378-8542999ddad1 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:33.746611 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b03c90-c463-48e1-8378-8542999ddad1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:33.746846 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:33.759889 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b03c90-c463-48e1-8378-8542999ddad1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:33.761409 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b03c90-c463-48e1-8378-8542999ddad1 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:33.777977 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-c9b03c90-c463-48e1-8378-8542999ddad1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:46:33.777977 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:33.777977 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:33.777977 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:33.777977 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:33.777977 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:33.777977 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:33.777977 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:33.777977 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:33.777977 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:33.777977 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:33.777977 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:33.777977 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 09:46:33.777977 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 09:46:33.777977 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 09:46:33.779015 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 09:46:33.779015 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:33.779015 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:33.779015 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:33.779015 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:33.779015 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:33.779015 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:33.779015 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:33.779015 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:33.779015 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:33.779015 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:33.779015 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:33.779015 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:33.779015 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:33.779015 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:33.779015 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:33.779015 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:33.780047 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 09:46:33.780047 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 09:46:33.780047 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:46:33.780047 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:46:33.780047 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 287/562] 10.222.0.64 () {56 vars in 1108 bytes} [Tue Aug 2 09:46:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:46:33.780994 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f26a9fc-c7bb-4043-9da1-4b9ff48fb82f None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:33.780994 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f26a9fc-c7bb-4043-9da1-4b9ff48fb82f None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:33.781129 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f26a9fc-c7bb-4043-9da1-4b9ff48fb82f None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:33.786784 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 288/563] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:46:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:46:33.786784 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f26a9fc-c7bb-4043-9da1-4b9ff48fb82f None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:33.806993 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f8a6f0-2948-46a7-8668-f868c243ae57 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:33.850832 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235c7559-8aa5-412e-800d-01f4b998433e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:33.851455 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235c7559-8aa5-412e-800d-01f4b998433e None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:33.851735 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235c7559-8aa5-412e-800d-01f4b998433e None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:33.852004 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235c7559-8aa5-412e-800d-01f4b998433e None admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:33.853503 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235c7559-8aa5-412e-800d-01f4b998433e None admin] RBAC: Adding query filter params (name=service) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:33.853809 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235c7559-8aa5-412e-800d-01f4b998433e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:33.855257 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235c7559-8aa5-412e-800d-01f4b998433e None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:33.864162 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 276/564] 10.222.0.64 () {56 vars in 1103 bytes} [Tue Aug 2 09:46:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:33.865037 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235c7559-8aa5-412e-800d-01f4b998433e None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:33.908664 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f26a9fc-c7bb-4043-9da1-4b9ff48fb82f None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:33.953874 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23beb3b9-632d-4a15-995c-46b2cd9846d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:33.954998 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23beb3b9-632d-4a15-995c-46b2cd9846d1 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:33.955289 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23beb3b9-632d-4a15-995c-46b2cd9846d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:33.955569 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23beb3b9-632d-4a15-995c-46b2cd9846d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:33.956411 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23beb3b9-632d-4a15-995c-46b2cd9846d1 None admin] RBAC: Adding query filter params (role.id=5704af09562a457fa35364e8af9481f1, scope.project.id=390bd7c4f91c4465b0c7ab0f9e436c4d, user.id=6a371e3df3764a4cbb1b8368aa74bd65) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:33.956732 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23beb3b9-632d-4a15-995c-46b2cd9846d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:33.958216 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23beb3b9-632d-4a15-995c-46b2cd9846d1 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:33.976468 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 289/565] 10.222.0.64 () {56 vars in 1374 bytes} [Tue Aug 2 09:46:33 2022] GET /identity/v3/role_assignments?user.id=6a371e3df3764a4cbb1b8368aa74bd65&scope.project.id=390bd7c4f91c4465b0c7ab0f9e436c4d&role.id=5704af09562a457fa35364e8af9481f1 => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:33.976468 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810f93ce-b2af-4450-a273-7a4a06678d5d None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:33.976468 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810f93ce-b2af-4450-a273-7a4a06678d5d None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:33.976468 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810f93ce-b2af-4450-a273-7a4a06678d5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:34.000045 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-810f93ce-b2af-4450-a273-7a4a06678d5d None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:46:34.001495 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-810f93ce-b2af-4450-a273-7a4a06678d5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:46:34.009773 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235c7559-8aa5-412e-800d-01f4b998433e None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:34.039315 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:34.060625 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:34.063149 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b286d6f-4164-4632-a0fa-73b8e9c8177f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:34.063958 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b286d6f-4164-4632-a0fa-73b8e9c8177f None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:34.064263 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b286d6f-4164-4632-a0fa-73b8e9c8177f None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:34.064577 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b286d6f-4164-4632-a0fa-73b8e9c8177f None admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:34.065505 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b286d6f-4164-4632-a0fa-73b8e9c8177f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:34.066264 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b286d6f-4164-4632-a0fa-73b8e9c8177f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:34.068028 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b286d6f-4164-4632-a0fa-73b8e9c8177f None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:34.090626 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 277/566] 10.222.0.64 () {56 vars in 1109 bytes} [Tue Aug 2 09:46:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:34.091971 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b286d6f-4164-4632-a0fa-73b8e9c8177f None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:34.099360 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 290/567] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:46:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 09:46:34.100647 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810f93ce-b2af-4450-a273-7a4a06678d5d None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:34.226119 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b286d6f-4164-4632-a0fa-73b8e9c8177f None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:34.226364 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810f93ce-b2af-4450-a273-7a4a06678d5d None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:34.263628 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d6dfdd-3912-44cc-b768-d66e2a080ccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:34.263824 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0379c6-cf7a-4a0b-932b-f706289d7882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:34.264390 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d6dfdd-3912-44cc-b768-d66e2a080ccc None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:34.264516 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0379c6-cf7a-4a0b-932b-f706289d7882 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:34.264593 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d6dfdd-3912-44cc-b768-d66e2a080ccc None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:34.264729 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0379c6-cf7a-4a0b-932b-f706289d7882 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:34.264863 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d6dfdd-3912-44cc-b768-d66e2a080ccc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:34.265186 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0379c6-cf7a-4a0b-932b-f706289d7882 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:34.274032 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0379c6-cf7a-4a0b-932b-f706289d7882 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:34.275576 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0379c6-cf7a-4a0b-932b-f706289d7882 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:34.276852 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d6dfdd-3912-44cc-b768-d66e2a080ccc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:34.278339 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d6dfdd-3912-44cc-b768-d66e2a080ccc None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:34.288372 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-de0379c6-cf7a-4a0b-932b-f706289d7882 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 02 09:46:34.288372 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:34.288372 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:34.288372 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:34.288372 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:34.288372 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:34.288372 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:34.288372 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:34.288372 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:34.288372 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:34.288372 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:34.288372 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:34.288372 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 09:46:34.288372 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 09:46:34.288372 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 09:46:34.288372 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 09:46:34.289616 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:34.289616 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:34.289616 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 09:46:34.289616 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 09:46:34.289616 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 09:46:34.289616 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 09:46:34.289616 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:34.289616 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:34.289616 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:34.289616 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:34.289616 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:34.289616 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:34.289616 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:34.289616 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:34.289616 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:34.289616 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:34.289616 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:34.289616 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:34.290793 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:34.290793 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:34.290793 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 09:46:34.290793 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 09:46:34.290793 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 09:46:34.290793 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 09:46:34.290793 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 09:46:34.290793 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 09:46:34.290793 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 02 09:46:34.290793 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:46:34.290793 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 291/568] 10.222.0.64 () {56 vars in 1099 bytes} [Tue Aug 2 09:46:34 2022] GET /identity/v3/users/cinder => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:46:34.290793 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdea1435-f72e-4be3-89a9-8dd4bbbb15b5 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:34.291597 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdea1435-f72e-4be3-89a9-8dd4bbbb15b5 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:34.291597 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdea1435-f72e-4be3-89a9-8dd4bbbb15b5 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:34.296645 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 292/569] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:46:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:46:34.296645 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdea1435-f72e-4be3-89a9-8dd4bbbb15b5 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:34.308668 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-77d6dfdd-3912-44cc-b768-d66e2a080ccc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:46:34.308668 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:34.308668 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:34.308668 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:34.308668 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:34.308668 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:34.308668 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:34.308668 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:34.308668 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:34.308668 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:34.308668 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:34.308668 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:34.308668 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 09:46:34.308668 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 09:46:34.308668 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 09:46:34.309935 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 09:46:34.309935 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:34.309935 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:34.309935 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:34.309935 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:34.309935 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:34.309935 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:34.309935 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:34.309935 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:34.309935 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:34.309935 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:34.309935 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:34.309935 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:34.309935 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:34.309935 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:34.309935 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:34.309935 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:34.310835 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 09:46:34.310835 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 09:46:34.310835 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:46:34.310835 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:46:34.310835 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 278/570] 10.222.0.64 () {56 vars in 1108 bytes} [Tue Aug 2 09:46:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:46:34.311492 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daf1cd5-5c3d-48fc-aac3-b1350cc6e076 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:34.311784 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daf1cd5-5c3d-48fc-aac3-b1350cc6e076 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:34.312064 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daf1cd5-5c3d-48fc-aac3-b1350cc6e076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:34.341915 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-5daf1cd5-5c3d-48fc-aac3-b1350cc6e076 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:46:34.343192 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-5daf1cd5-5c3d-48fc-aac3-b1350cc6e076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:46:34.353718 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:34.376842 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:34.429848 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 279/571] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:46:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 09:46:34.430884 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daf1cd5-5c3d-48fc-aac3-b1350cc6e076 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:34.444475 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdea1435-f72e-4be3-89a9-8dd4bbbb15b5 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:34.483517 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c053f18c-beef-41c9-ae46-0ad02ababd31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:34.484373 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c053f18c-beef-41c9-ae46-0ad02ababd31 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:34.484746 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c053f18c-beef-41c9-ae46-0ad02ababd31 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:34.484997 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c053f18c-beef-41c9-ae46-0ad02ababd31 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:34.504209 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c053f18c-beef-41c9-ae46-0ad02ababd31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 09:46:34.512312 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-c053f18c-beef-41c9-ae46-0ad02ababd31 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 09:46:34.512312 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:34.512312 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:34.512312 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:34.512312 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:34.512312 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:34.512312 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:34.512312 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:34.512312 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:34.512312 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:34.512312 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:34.512312 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:34.512312 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 09:46:34.512312 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 09:46:34.512312 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 09:46:34.512312 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 09:46:34.513505 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 09:46:34.513505 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 09:46:34.513505 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 09:46:34.513505 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 09:46:34.513505 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:34.513505 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:34.513505 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:34.513505 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:34.513505 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:34.513505 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:34.513505 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:34.513505 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:34.513505 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:34.513505 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:34.513505 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:34.513505 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:34.514605 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:34.514605 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:34.514605 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:34.514605 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:34.514605 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 09:46:34.514605 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 09:46:34.514605 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 09:46:34.514605 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 09:46:34.514605 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 09:46:34.514605 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 09:46:34.514605 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 09:46:34.514605 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:46:34.514605 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 293/572] 10.222.0.64 () {56 vars in 1102 bytes} [Tue Aug 2 09:46:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:46:34.514605 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c053f18c-beef-41c9-ae46-0ad02ababd31 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:34.590352 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daf1cd5-5c3d-48fc-aac3-b1350cc6e076 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:34.628860 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45219da-1660-4580-8b2e-658aec8794da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:34.629491 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45219da-1660-4580-8b2e-658aec8794da None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:34.629773 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45219da-1660-4580-8b2e-658aec8794da None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:34.630048 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45219da-1660-4580-8b2e-658aec8794da None admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:34.630853 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45219da-1660-4580-8b2e-658aec8794da None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:34.631172 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45219da-1660-4580-8b2e-658aec8794da None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:34.632633 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45219da-1660-4580-8b2e-658aec8794da None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:34.633373 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:34.650840 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c053f18c-beef-41c9-ae46-0ad02ababd31 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:34.656647 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 280/573] 10.222.0.64 () {56 vars in 1137 bytes} [Tue Aug 2 09:46:34 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:34.657429 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45219da-1660-4580-8b2e-658aec8794da None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:34.683748 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:34.703671 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104e5b61-09db-41ab-827b-ac9fe30b2f85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:34.704941 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104e5b61-09db-41ab-827b-ac9fe30b2f85 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:34.705235 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104e5b61-09db-41ab-827b-ac9fe30b2f85 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:34.705513 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104e5b61-09db-41ab-827b-ac9fe30b2f85 None admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:34.706317 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104e5b61-09db-41ab-827b-ac9fe30b2f85 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:34.706633 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104e5b61-09db-41ab-827b-ac9fe30b2f85 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:34.708035 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104e5b61-09db-41ab-827b-ac9fe30b2f85 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:34.727210 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 294/574] 10.222.0.64 () {56 vars in 1109 bytes} [Tue Aug 2 09:46:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:34.727210 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104e5b61-09db-41ab-827b-ac9fe30b2f85 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:34.799844 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45219da-1660-4580-8b2e-658aec8794da None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:34.852825 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d6dbc4-c752-4eff-a827-5c40dceb95a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:34.853478 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d6dbc4-c752-4eff-a827-5c40dceb95a4 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:34.853759 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d6dbc4-c752-4eff-a827-5c40dceb95a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:34.854030 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d6dbc4-c752-4eff-a827-5c40dceb95a4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:34.860171 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104e5b61-09db-41ab-827b-ac9fe30b2f85 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:34.865312 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d6dbc4-c752-4eff-a827-5c40dceb95a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 09:46:34.870237 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-02d6dbc4-c752-4eff-a827-5c40dceb95a4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 09:46:34.870237 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:34.870237 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:34.870237 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:34.870237 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:34.870237 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:34.870237 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:34.870237 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:34.870237 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:34.870237 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:34.870237 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:34.870237 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:34.870237 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 09:46:34.870237 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 09:46:34.870237 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 09:46:34.870237 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 09:46:34.871186 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 09:46:34.871186 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 09:46:34.871186 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 09:46:34.871186 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 09:46:34.871186 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:34.871186 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:34.871186 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:34.871186 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:34.871186 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:34.871186 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:34.871186 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:34.871186 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:34.871186 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:34.871186 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:34.871186 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:34.871186 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:34.877819 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:34.877819 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:34.877819 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:34.877819 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:34.877819 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 09:46:34.877819 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 09:46:34.877819 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 09:46:34.877819 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 09:46:34.877819 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 09:46:34.877819 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 09:46:34.877819 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 09:46:34.877819 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:46:34.877819 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 281/575] 10.222.0.64 () {56 vars in 1102 bytes} [Tue Aug 2 09:46:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:46:34.877819 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d6dbc4-c752-4eff-a827-5c40dceb95a4 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:34.912687 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebaf30c-fc1e-4761-a195-e8b27481671d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:34.913507 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebaf30c-fc1e-4761-a195-e8b27481671d None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:34.913858 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebaf30c-fc1e-4761-a195-e8b27481671d None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:34.914244 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebaf30c-fc1e-4761-a195-e8b27481671d None admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:34.915041 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebaf30c-fc1e-4761-a195-e8b27481671d None admin] RBAC: Adding query filter params (name=service) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:34.915398 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebaf30c-fc1e-4761-a195-e8b27481671d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:34.916906 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebaf30c-fc1e-4761-a195-e8b27481671d None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:34.925625 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 295/576] 10.222.0.64 () {56 vars in 1103 bytes} [Tue Aug 2 09:46:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:34.928633 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebaf30c-fc1e-4761-a195-e8b27481671d None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:34.954679 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:34.980527 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:35.015511 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d6dbc4-c752-4eff-a827-5c40dceb95a4 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:35.068447 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebaf30c-fc1e-4761-a195-e8b27481671d None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:35.073698 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe6b69c-036f-45b5-9461-add6b580d8e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:35.073698 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe6b69c-036f-45b5-9461-add6b580d8e5 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:35.073698 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe6b69c-036f-45b5-9461-add6b580d8e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:35.073698 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe6b69c-036f-45b5-9461-add6b580d8e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:35.084815 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe6b69c-036f-45b5-9461-add6b580d8e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:35.086301 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe6b69c-036f-45b5-9461-add6b580d8e5 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:35.112544 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-6fe6b69c-036f-45b5-9461-add6b580d8e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:46:35.112544 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:35.112544 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:35.112544 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:35.112544 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:35.112544 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:35.112544 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:35.112544 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:35.112544 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:35.112544 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:35.112544 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:35.112544 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:35.112544 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 09:46:35.112544 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 09:46:35.112544 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 09:46:35.114331 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 09:46:35.114331 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:35.114331 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:35.114331 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:35.114331 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:35.114331 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:35.114331 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:35.114331 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:35.114331 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:35.114331 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:35.114331 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:35.114331 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:35.114331 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:35.114331 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:35.114331 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:35.114331 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:35.114331 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:35.115381 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 09:46:35.115381 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 09:46:35.115381 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:46:35.115381 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:46:35.115381 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 282/577] 10.222.0.64 () {56 vars in 1108 bytes} [Tue Aug 2 09:46:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:46:35.115381 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe6b69c-036f-45b5-9461-add6b580d8e5 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:35.131298 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe7a0e5-52e4-4431-bb7a-65d1c8fa294a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:35.131971 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe7a0e5-52e4-4431-bb7a-65d1c8fa294a None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:35.132292 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe7a0e5-52e4-4431-bb7a-65d1c8fa294a None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:35.132555 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe7a0e5-52e4-4431-bb7a-65d1c8fa294a None admin] PATH_INFO: `/v3/users/nova` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:35.141008 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe7a0e5-52e4-4431-bb7a-65d1c8fa294a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:35.143052 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe7a0e5-52e4-4431-bb7a-65d1c8fa294a None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:35.154431 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-fbe7a0e5-52e4-4431-bb7a-65d1c8fa294a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 02 09:46:35.154431 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:35.154431 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:35.154431 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:35.154431 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:35.154431 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:35.154431 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:35.154431 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:35.154431 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:35.154431 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:35.154431 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:35.154431 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:35.154431 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 09:46:35.154431 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 09:46:35.154431 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 09:46:35.154431 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 09:46:35.155411 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:35.155411 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:35.155411 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 09:46:35.155411 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 09:46:35.155411 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 09:46:35.155411 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 09:46:35.155411 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:35.155411 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:35.155411 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:35.155411 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:35.155411 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:35.155411 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:35.155411 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:35.155411 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:35.155411 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:35.155411 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:35.155411 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:35.155411 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:35.161483 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:35.161483 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:35.161483 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 09:46:35.161483 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 09:46:35.161483 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 09:46:35.161483 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 09:46:35.161483 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 09:46:35.161483 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 09:46:35.161483 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 02 09:46:35.161483 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:46:35.161483 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 296/578] 10.222.0.64 () {56 vars in 1093 bytes} [Tue Aug 2 09:46:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:46:35.161483 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe7a0e5-52e4-4431-bb7a-65d1c8fa294a None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:35.246825 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe6b69c-036f-45b5-9461-add6b580d8e5 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:35.267281 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:35.282893 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:35.289938 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e14e68-88c9-4e1a-b99d-73e6cd4bb1f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:35.290778 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e14e68-88c9-4e1a-b99d-73e6cd4bb1f6 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:35.291229 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e14e68-88c9-4e1a-b99d-73e6cd4bb1f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:35.291661 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e14e68-88c9-4e1a-b99d-73e6cd4bb1f6 None admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:35.292437 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e14e68-88c9-4e1a-b99d-73e6cd4bb1f6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:35.292748 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e14e68-88c9-4e1a-b99d-73e6cd4bb1f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:35.294136 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e14e68-88c9-4e1a-b99d-73e6cd4bb1f6 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:35.297574 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe7a0e5-52e4-4431-bb7a-65d1c8fa294a None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:35.308712 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 283/579] 10.222.0.64 () {56 vars in 1103 bytes} [Tue Aug 2 09:46:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:35.309420 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e14e68-88c9-4e1a-b99d-73e6cd4bb1f6 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:35.339497 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06db848-7fec-41be-85b4-aae09f393365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:35.340059 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06db848-7fec-41be-85b4-aae09f393365 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:35.340407 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06db848-7fec-41be-85b4-aae09f393365 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:35.340674 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06db848-7fec-41be-85b4-aae09f393365 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:35.352238 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06db848-7fec-41be-85b4-aae09f393365 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:35.353878 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06db848-7fec-41be-85b4-aae09f393365 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:35.369626 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-c06db848-7fec-41be-85b4-aae09f393365 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:46:35.369626 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:35.369626 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:35.369626 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:35.369626 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:35.369626 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:35.369626 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:35.369626 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:35.369626 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:35.369626 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:35.369626 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:35.369626 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:35.369626 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 09:46:35.369626 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 09:46:35.369626 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 09:46:35.370590 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 09:46:35.370590 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:35.370590 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:35.370590 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:35.370590 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:35.370590 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:35.370590 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:35.370590 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:35.370590 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:35.370590 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:35.370590 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:35.370590 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:35.370590 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:35.370590 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:35.370590 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:35.370590 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:35.370590 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:35.371496 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 09:46:35.371496 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 09:46:35.371496 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:46:35.371496 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:46:35.371496 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 297/580] 10.222.0.64 () {56 vars in 1108 bytes} [Tue Aug 2 09:46:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:46:35.371773 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06db848-7fec-41be-85b4-aae09f393365 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:35.433676 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e14e68-88c9-4e1a-b99d-73e6cd4bb1f6 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:35.480153 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4c1c99-c3f0-4a44-b582-a468bc4946d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:35.481116 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4c1c99-c3f0-4a44-b582-a468bc4946d1 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:35.481606 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4c1c99-c3f0-4a44-b582-a468bc4946d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:35.482063 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4c1c99-c3f0-4a44-b582-a468bc4946d1 None admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:35.482969 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4c1c99-c3f0-4a44-b582-a468bc4946d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:35.483472 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4c1c99-c3f0-4a44-b582-a468bc4946d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:35.485043 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4c1c99-c3f0-4a44-b582-a468bc4946d1 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:35.490228 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06db848-7fec-41be-85b4-aae09f393365 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:35.497871 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 284/581] 10.222.0.64 () {56 vars in 1109 bytes} [Tue Aug 2 09:46:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:35.498931 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4c1c99-c3f0-4a44-b582-a468bc4946d1 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:35.523637 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:35.534200 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56aab71d-a4cd-4830-82d9-1f04ecb11e1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:35.535109 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56aab71d-a4cd-4830-82d9-1f04ecb11e1b None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:35.535583 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56aab71d-a4cd-4830-82d9-1f04ecb11e1b None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:35.535865 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56aab71d-a4cd-4830-82d9-1f04ecb11e1b None admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:35.536707 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56aab71d-a4cd-4830-82d9-1f04ecb11e1b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:35.537019 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56aab71d-a4cd-4830-82d9-1f04ecb11e1b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:35.538487 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56aab71d-a4cd-4830-82d9-1f04ecb11e1b None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:35.565990 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 298/582] 10.222.0.64 () {56 vars in 1133 bytes} [Tue Aug 2 09:46:35 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:35.566189 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:35.566288 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56aab71d-a4cd-4830-82d9-1f04ecb11e1b None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:35.625619 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4c1c99-c3f0-4a44-b582-a468bc4946d1 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:35.677274 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dc9f3e-aff8-4fe8-84e7-110fb1c49458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:35.678108 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dc9f3e-aff8-4fe8-84e7-110fb1c49458 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:35.678403 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dc9f3e-aff8-4fe8-84e7-110fb1c49458 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:35.678752 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dc9f3e-aff8-4fe8-84e7-110fb1c49458 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:35.695879 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dc9f3e-aff8-4fe8-84e7-110fb1c49458 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:35.697409 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dc9f3e-aff8-4fe8-84e7-110fb1c49458 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:35.699186 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56aab71d-a4cd-4830-82d9-1f04ecb11e1b None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:35.714594 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-f9dc9f3e-aff8-4fe8-84e7-110fb1c49458 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:46:35.714594 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:35.714594 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:35.714594 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:35.714594 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:35.714594 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:35.714594 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:35.714594 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:35.714594 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:35.714594 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:35.714594 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:35.714594 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:35.714594 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 09:46:35.714594 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 09:46:35.714594 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 09:46:35.715810 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 09:46:35.715810 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:35.715810 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:35.715810 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:35.715810 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:35.715810 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:35.715810 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:35.715810 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:35.715810 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:35.715810 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:35.715810 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:35.715810 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:35.715810 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:35.715810 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:35.715810 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:35.715810 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:35.715810 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:35.717261 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 09:46:35.717261 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 09:46:35.717261 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:46:35.717261 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:46:35.717261 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 285/583] 10.222.0.64 () {56 vars in 1108 bytes} [Tue Aug 2 09:46:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:46:35.717261 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dc9f3e-aff8-4fe8-84e7-110fb1c49458 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:35.747221 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57075580-30a3-471e-91b1-b056d95c7246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:35.747859 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57075580-30a3-471e-91b1-b056d95c7246 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:35.748159 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57075580-30a3-471e-91b1-b056d95c7246 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:35.748470 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57075580-30a3-471e-91b1-b056d95c7246 None admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:35.749164 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57075580-30a3-471e-91b1-b056d95c7246 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:35.749479 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57075580-30a3-471e-91b1-b056d95c7246 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:35.751592 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57075580-30a3-471e-91b1-b056d95c7246 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:35.765704 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 299/584] 10.222.0.64 () {56 vars in 1109 bytes} [Tue Aug 2 09:46:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:35.767179 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57075580-30a3-471e-91b1-b056d95c7246 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:35.820828 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:35.842714 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dc9f3e-aff8-4fe8-84e7-110fb1c49458 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:35.856182 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:35.888090 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eeec1ae-7a29-4fad-9294-830e1be69f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:35.889280 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeec1ae-7a29-4fad-9294-830e1be69f5c None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:35.889280 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeec1ae-7a29-4fad-9294-830e1be69f5c None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:35.889280 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeec1ae-7a29-4fad-9294-830e1be69f5c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:35.897515 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eeec1ae-7a29-4fad-9294-830e1be69f5c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:35.898973 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eeec1ae-7a29-4fad-9294-830e1be69f5c None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:35.900028 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57075580-30a3-471e-91b1-b056d95c7246 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:35.910976 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-5eeec1ae-7a29-4fad-9294-830e1be69f5c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 09:46:35.910976 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:35.910976 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:35.910976 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:35.910976 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:35.910976 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:35.910976 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:35.910976 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:35.910976 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:35.910976 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:35.910976 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:35.910976 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:35.910976 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 09:46:35.910976 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 09:46:35.910976 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 09:46:35.912126 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 09:46:35.912126 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:35.912126 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:35.912126 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:35.912126 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:35.912126 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:35.912126 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:35.912126 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:35.912126 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:35.912126 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:35.912126 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:35.912126 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:35.912126 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:35.912126 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:35.912126 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:35.912126 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:35.912126 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:35.913055 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 09:46:35.913055 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 09:46:35.913055 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 09:46:35.913055 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 09:46:35.913055 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 09:46:35.913055 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 09:46:35.913055 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 09:46:35.913055 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:46:35.913055 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 286/585] 10.222.0.64 () {56 vars in 1111 bytes} [Tue Aug 2 09:46:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:46:35.913552 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eeec1ae-7a29-4fad-9294-830e1be69f5c None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:35.961433 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b86492f-65cb-448c-ae0d-c914c4b79623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:35.962410 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b86492f-65cb-448c-ae0d-c914c4b79623 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:35.962879 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b86492f-65cb-448c-ae0d-c914c4b79623 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:35.963341 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b86492f-65cb-448c-ae0d-c914c4b79623 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:35.975297 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b86492f-65cb-448c-ae0d-c914c4b79623 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:35.977095 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b86492f-65cb-448c-ae0d-c914c4b79623 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:35.996112 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-9b86492f-65cb-448c-ae0d-c914c4b79623 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:46:35.996112 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:35.996112 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:35.996112 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:35.996112 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:35.996112 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:35.996112 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:35.996112 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:35.996112 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:35.996112 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:35.996112 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:35.996112 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:35.996112 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 09:46:35.996112 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 09:46:35.996112 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 09:46:35.997280 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 09:46:35.997280 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:35.997280 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:35.997280 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:35.997280 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:35.997280 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:35.997280 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:35.997280 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:35.997280 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:35.997280 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:35.997280 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:35.997280 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:35.997280 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:35.997280 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:35.997280 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:35.997280 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:35.997280 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:35.998385 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 09:46:35.998385 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 09:46:35.998385 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:46:35.998385 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:46:35.999962 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 300/586] 10.222.0.64 () {56 vars in 1108 bytes} [Tue Aug 2 09:46:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:46:36.001718 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b86492f-65cb-448c-ae0d-c914c4b79623 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:36.049149 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eeec1ae-7a29-4fad-9294-830e1be69f5c None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:36.090174 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f084fc-f115-41e0-92e7-8a4ee1e015bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:36.090846 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f084fc-f115-41e0-92e7-8a4ee1e015bc None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:36.091076 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f084fc-f115-41e0-92e7-8a4ee1e015bc None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:36.091357 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f084fc-f115-41e0-92e7-8a4ee1e015bc None admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:36.092065 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f084fc-f115-41e0-92e7-8a4ee1e015bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:36.092414 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f084fc-f115-41e0-92e7-8a4ee1e015bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:36.093801 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f084fc-f115-41e0-92e7-8a4ee1e015bc None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:36.105698 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:36.106369 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 287/587] 10.222.0.64 () {56 vars in 1109 bytes} [Tue Aug 2 09:46:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:36.107026 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f084fc-f115-41e0-92e7-8a4ee1e015bc None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:36.112293 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b86492f-65cb-448c-ae0d-c914c4b79623 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:36.140399 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:36.154898 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2e534c-f571-478d-91b2-ed5ccceeef64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:36.155547 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2e534c-f571-478d-91b2-ed5ccceeef64 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:36.155840 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2e534c-f571-478d-91b2-ed5ccceeef64 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:36.156127 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2e534c-f571-478d-91b2-ed5ccceeef64 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:36.164882 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2e534c-f571-478d-91b2-ed5ccceeef64 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:36.166349 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2e534c-f571-478d-91b2-ed5ccceeef64 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:36.178681 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-0f2e534c-f571-478d-91b2-ed5ccceeef64 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 02 09:46:36.178681 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:36.178681 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:36.178681 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:36.178681 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:36.178681 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:36.178681 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:36.178681 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:36.178681 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:36.178681 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:36.178681 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:36.178681 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:36.178681 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 09:46:36.178681 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 09:46:36.178681 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 09:46:36.178681 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 09:46:36.179773 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:36.179773 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:36.179773 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 09:46:36.179773 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 09:46:36.179773 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 09:46:36.179773 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 09:46:36.179773 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:36.179773 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:36.179773 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:36.179773 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:36.179773 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:36.179773 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:36.179773 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:36.179773 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:36.179773 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:36.179773 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:36.179773 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:36.179773 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:36.180986 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:36.180986 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:36.180986 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 09:46:36.180986 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 09:46:36.180986 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 09:46:36.180986 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 09:46:36.180986 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 09:46:36.180986 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 09:46:36.180986 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 02 09:46:36.180986 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:46:36.180986 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 301/588] 10.222.0.64 () {56 vars in 1099 bytes} [Tue Aug 2 09:46:36 2022] GET /identity/v3/users/glance => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:46:36.181587 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2e534c-f571-478d-91b2-ed5ccceeef64 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:36.218477 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f084fc-f115-41e0-92e7-8a4ee1e015bc None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:36.254441 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cdaabb-4af4-4425-b4f6-cbf61c15c1c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:36.255064 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cdaabb-4af4-4425-b4f6-cbf61c15c1c1 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:36.255344 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cdaabb-4af4-4425-b4f6-cbf61c15c1c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:36.255615 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cdaabb-4af4-4425-b4f6-cbf61c15c1c1 None admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:36.256401 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cdaabb-4af4-4425-b4f6-cbf61c15c1c1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:36.256711 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cdaabb-4af4-4425-b4f6-cbf61c15c1c1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:36.258137 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cdaabb-4af4-4425-b4f6-cbf61c15c1c1 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:36.270352 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 288/589] 10.222.0.64 () {56 vars in 1148 bytes} [Tue Aug 2 09:46:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:36.271695 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a40e1f-f8f8-4669-8f86-fb49e7bc8b8e None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:36.272029 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a40e1f-f8f8-4669-8f86-fb49e7bc8b8e None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:36.272272 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a40e1f-f8f8-4669-8f86-fb49e7bc8b8e None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:36.273809 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 289/590] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:46:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:46:36.276494 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a40e1f-f8f8-4669-8f86-fb49e7bc8b8e None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:36.295319 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2e534c-f571-478d-91b2-ed5ccceeef64 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:36.336612 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a863ed-d4bf-499d-af7c-f90c9c729222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:36.337241 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a863ed-d4bf-499d-af7c-f90c9c729222 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:36.337530 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a863ed-d4bf-499d-af7c-f90c9c729222 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:36.337810 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a863ed-d4bf-499d-af7c-f90c9c729222 None admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:36.338513 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a863ed-d4bf-499d-af7c-f90c9c729222 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:36.338849 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a863ed-d4bf-499d-af7c-f90c9c729222 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:36.340230 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a863ed-d4bf-499d-af7c-f90c9c729222 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:36.353439 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 302/591] 10.222.0.64 () {56 vars in 1109 bytes} [Tue Aug 2 09:46:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:36.354293 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a863ed-d4bf-499d-af7c-f90c9c729222 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:36.379493 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:36.388680 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:36.408793 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a40e1f-f8f8-4669-8f86-fb49e7bc8b8e None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:36.448445 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4aef4f1-fedb-4980-9c59-322da22fd4e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:36.449090 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aef4f1-fedb-4980-9c59-322da22fd4e3 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:36.449380 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aef4f1-fedb-4980-9c59-322da22fd4e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:36.449654 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aef4f1-fedb-4980-9c59-322da22fd4e3 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:36.457599 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4aef4f1-fedb-4980-9c59-322da22fd4e3 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:36.459057 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4aef4f1-fedb-4980-9c59-322da22fd4e3 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:36.465615 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a863ed-d4bf-499d-af7c-f90c9c729222 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:36.471196 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-e4aef4f1-fedb-4980-9c59-322da22fd4e3 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 02 09:46:36.471196 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:36.471196 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:36.471196 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:36.471196 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:36.471196 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:36.471196 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:36.471196 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:36.471196 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:36.471196 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:36.471196 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:36.471196 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:36.471196 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 09:46:36.471196 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 09:46:36.471196 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 09:46:36.471196 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 09:46:36.472394 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:36.472394 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:36.472394 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 09:46:36.472394 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 09:46:36.472394 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 09:46:36.472394 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 09:46:36.472394 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:36.472394 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:36.472394 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:36.472394 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:36.472394 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:36.472394 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:36.472394 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:36.472394 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:36.472394 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:36.472394 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:36.472394 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:36.472394 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:36.473364 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:36.473364 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:36.473364 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 09:46:36.473364 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 09:46:36.473364 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 09:46:36.473364 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 09:46:36.473364 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 09:46:36.473364 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 09:46:36.473364 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 02 09:46:36.473364 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:46:36.473364 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 290/592] 10.222.0.64 () {56 vars in 1102 bytes} [Tue Aug 2 09:46:36 2022] GET /identity/v3/users/neutron => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:46:36.473364 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4aef4f1-fedb-4980-9c59-322da22fd4e3 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:36.506529 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa591ec7-e100-4d4f-a5eb-e16971735c19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:36.507172 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa591ec7-e100-4d4f-a5eb-e16971735c19 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:36.507463 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa591ec7-e100-4d4f-a5eb-e16971735c19 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:36.507734 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa591ec7-e100-4d4f-a5eb-e16971735c19 None admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:36.508573 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa591ec7-e100-4d4f-a5eb-e16971735c19 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:36.508884 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa591ec7-e100-4d4f-a5eb-e16971735c19 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:36.510316 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa591ec7-e100-4d4f-a5eb-e16971735c19 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:36.528749 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 303/593] 10.222.0.64 () {56 vars in 1137 bytes} [Tue Aug 2 09:46:36 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:36.529922 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b836522-4de0-4777-a2b2-1f64ab17542f None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:36.530213 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b836522-4de0-4777-a2b2-1f64ab17542f None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:36.530502 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b836522-4de0-4777-a2b2-1f64ab17542f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:36.549772 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-4b836522-4de0-4777-a2b2-1f64ab17542f None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:46:36.550984 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-4b836522-4de0-4777-a2b2-1f64ab17542f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:46:36.579579 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4aef4f1-fedb-4980-9c59-322da22fd4e3 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:36.616324 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ac45b2-c2f1-4ed7-ac42-e76b05e05feb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:36.616996 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ac45b2-c2f1-4ed7-ac42-e76b05e05feb None admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:36.617278 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ac45b2-c2f1-4ed7-ac42-e76b05e05feb None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:36.617555 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ac45b2-c2f1-4ed7-ac42-e76b05e05feb None admin] PATH_INFO: `/v3/projects/390bd7c4f91c4465b0c7ab0f9e436c4d/users/10d7de62b9ee4039bdc79627ac7fb499/roles/5704af09562a457fa35364e8af9481f1` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:36.630777 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 304/594] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:46:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 09:46:36.631712 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b836522-4de0-4777-a2b2-1f64ab17542f None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:36.634471 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ac45b2-c2f1-4ed7-ac42-e76b05e05feb None admin] RBAC: Authorizing `identity:create_grant(project_id=390bd7c4f91c4465b0c7ab0f9e436c4d, user_id=10d7de62b9ee4039bdc79627ac7fb499, role_id=5704af09562a457fa35364e8af9481f1)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:36.637435 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ac45b2-c2f1-4ed7-ac42-e76b05e05feb None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:36.640090 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:36.640629 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:36.653990 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 291/595] 10.222.0.64 () {58 vars in 1439 bytes} [Tue Aug 2 09:46:36 2022] PUT /identity/v3/projects/390bd7c4f91c4465b0c7ab0f9e436c4d/users/10d7de62b9ee4039bdc79627ac7fb499/roles/5704af09562a457fa35364e8af9481f1 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:46:36.654902 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ac45b2-c2f1-4ed7-ac42-e76b05e05feb None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:36.769712 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b836522-4de0-4777-a2b2-1f64ab17542f None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:36.801517 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ac45b2-c2f1-4ed7-ac42-e76b05e05feb None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:36.815766 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83acc9b2-86b1-4530-9b44-60fc0a4a7f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:36.816840 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83acc9b2-86b1-4530-9b44-60fc0a4a7f21 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:36.817360 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83acc9b2-86b1-4530-9b44-60fc0a4a7f21 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:36.817845 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83acc9b2-86b1-4530-9b44-60fc0a4a7f21 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:36.827034 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83acc9b2-86b1-4530-9b44-60fc0a4a7f21 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:36.828565 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83acc9b2-86b1-4530-9b44-60fc0a4a7f21 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:36.842256 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-83acc9b2-86b1-4530-9b44-60fc0a4a7f21 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 09:46:36.842256 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:36.842256 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:36.842256 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:36.842256 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:36.842256 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:36.842256 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:36.842256 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:36.842256 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:36.842256 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:36.842256 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:36.842256 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:36.842256 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 09:46:36.842256 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 09:46:36.842256 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 09:46:36.843218 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 09:46:36.843218 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:36.843218 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:36.843218 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:36.843218 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:36.843218 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:36.843218 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:36.843218 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:36.843218 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:36.843218 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:36.843218 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:36.843218 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:36.843218 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:36.843218 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:36.843218 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:36.843218 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:36.843218 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:36.844149 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 09:46:36.844149 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 09:46:36.844149 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 09:46:36.844149 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 09:46:36.844149 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 09:46:36.844149 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 09:46:36.844149 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 09:46:36.844149 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:46:36.844149 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 305/596] 10.222.0.64 () {56 vars in 1111 bytes} [Tue Aug 2 09:46:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:46:36.845262 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83acc9b2-86b1-4530-9b44-60fc0a4a7f21 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:36.854978 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdb1406-224a-48c3-a14e-c2354923b770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:36.855612 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdb1406-224a-48c3-a14e-c2354923b770 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:36.855897 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdb1406-224a-48c3-a14e-c2354923b770 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:36.856180 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdb1406-224a-48c3-a14e-c2354923b770 None admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:36.857053 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdb1406-224a-48c3-a14e-c2354923b770 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:36.857357 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdb1406-224a-48c3-a14e-c2354923b770 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:36.858988 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdb1406-224a-48c3-a14e-c2354923b770 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:36.880729 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 292/597] 10.222.0.64 () {56 vars in 1139 bytes} [Tue Aug 2 09:46:36 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:36.881904 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdb1406-224a-48c3-a14e-c2354923b770 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:36.932200 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:36.952081 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:36.981161 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83acc9b2-86b1-4530-9b44-60fc0a4a7f21 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:37.015539 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdb1406-224a-48c3-a14e-c2354923b770 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:37.022054 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe975984-a61e-4f52-b115-0a10aade5d9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:37.022636 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe975984-a61e-4f52-b115-0a10aade5d9c None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:37.022925 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe975984-a61e-4f52-b115-0a10aade5d9c None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:37.023203 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe975984-a61e-4f52-b115-0a10aade5d9c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:37.037920 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe975984-a61e-4f52-b115-0a10aade5d9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:37.038126 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe975984-a61e-4f52-b115-0a10aade5d9c None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:37.056565 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655e4e61-a40b-4330-ace4-c57dfbf79f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:37.057213 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655e4e61-a40b-4330-ace4-c57dfbf79f8a None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:37.057498 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655e4e61-a40b-4330-ace4-c57dfbf79f8a None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:37.057597 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-fe975984-a61e-4f52-b115-0a10aade5d9c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:46:37.057597 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:37.057597 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:37.057597 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:37.057597 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:37.057597 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:37.057597 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:37.057597 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:37.057597 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:37.057597 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:37.057597 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:37.057597 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:37.057597 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 09:46:37.057597 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 09:46:37.057597 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 09:46:37.058445 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 09:46:37.058445 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:37.058445 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:37.058445 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:37.058445 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:37.058445 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:37.058445 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:37.058445 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:37.058445 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:37.058445 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:37.058445 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:37.058445 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:37.058445 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:37.058445 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:37.058445 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:37.058445 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:37.058445 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:37.059370 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 09:46:37.059370 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 09:46:37.059370 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:46:37.059370 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:46:37.059608 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655e4e61-a40b-4330-ace4-c57dfbf79f8a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:37.059687 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 306/598] 10.222.0.64 () {56 vars in 1108 bytes} [Tue Aug 2 09:46:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:46:37.059687 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe975984-a61e-4f52-b115-0a10aade5d9c None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:37.074061 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655e4e61-a40b-4330-ace4-c57dfbf79f8a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 09:46:37.079627 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-655e4e61-a40b-4330-ace4-c57dfbf79f8a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 09:46:37.079627 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:37.079627 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:37.079627 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:37.079627 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:37.079627 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:37.079627 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:37.079627 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:37.079627 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:37.079627 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:37.079627 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:37.079627 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:37.079627 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 09:46:37.079627 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 09:46:37.079627 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 09:46:37.079627 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 09:46:37.080753 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 09:46:37.080753 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 09:46:37.080753 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 09:46:37.080753 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 09:46:37.080753 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:37.080753 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:37.080753 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:37.080753 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:37.080753 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:37.080753 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:37.080753 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:37.080753 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:37.080753 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:37.080753 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:37.080753 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:37.080753 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:37.081626 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:37.081626 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:37.081626 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:37.081626 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:37.081626 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 09:46:37.081626 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 09:46:37.081626 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 09:46:37.081626 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 09:46:37.081626 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 09:46:37.081626 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 09:46:37.081626 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 09:46:37.081626 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:46:37.081626 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 293/599] 10.222.0.64 () {56 vars in 1102 bytes} [Tue Aug 2 09:46:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:46:37.081626 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655e4e61-a40b-4330-ace4-c57dfbf79f8a None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:37.194061 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe975984-a61e-4f52-b115-0a10aade5d9c None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:37.208151 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655e4e61-a40b-4330-ace4-c57dfbf79f8a None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:37.217309 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:37.227240 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:37.238226 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8c4bee-5e6e-4bbd-a948-c18220aee1e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:37.239022 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8c4bee-5e6e-4bbd-a948-c18220aee1e9 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:37.239316 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8c4bee-5e6e-4bbd-a948-c18220aee1e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:37.239598 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8c4bee-5e6e-4bbd-a948-c18220aee1e9 None admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:37.240431 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8c4bee-5e6e-4bbd-a948-c18220aee1e9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:37.240756 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8c4bee-5e6e-4bbd-a948-c18220aee1e9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:37.242209 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8c4bee-5e6e-4bbd-a948-c18220aee1e9 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:37.255876 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 307/600] 10.222.0.64 () {56 vars in 1148 bytes} [Tue Aug 2 09:46:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:37.257084 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8c4bee-5e6e-4bbd-a948-c18220aee1e9 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:37.262972 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80092819-c2f0-4540-a341-9302e807b6be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:37.264499 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80092819-c2f0-4540-a341-9302e807b6be None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:37.265238 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80092819-c2f0-4540-a341-9302e807b6be None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:37.265973 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80092819-c2f0-4540-a341-9302e807b6be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:37.286829 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80092819-c2f0-4540-a341-9302e807b6be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:37.288533 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80092819-c2f0-4540-a341-9302e807b6be None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:37.308601 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-80092819-c2f0-4540-a341-9302e807b6be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:46:37.308601 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:37.308601 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:37.308601 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:37.308601 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:37.308601 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:37.308601 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:37.308601 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:37.308601 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:37.308601 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:37.308601 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:37.308601 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:37.308601 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 09:46:37.308601 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 09:46:37.308601 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 09:46:37.309677 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 09:46:37.309677 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:37.309677 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:37.309677 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:37.309677 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:37.309677 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:37.309677 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:37.309677 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:37.309677 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:37.309677 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:37.309677 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:37.309677 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:37.309677 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:37.309677 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:37.309677 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:37.309677 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:37.309677 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:37.310575 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 09:46:37.310575 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 09:46:37.310575 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:46:37.310575 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:46:37.310575 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 294/601] 10.222.0.64 () {56 vars in 1108 bytes} [Tue Aug 2 09:46:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:46:37.311472 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80092819-c2f0-4540-a341-9302e807b6be None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:37.393475 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8c4bee-5e6e-4bbd-a948-c18220aee1e9 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:37.429949 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80092819-c2f0-4540-a341-9302e807b6be None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:37.437609 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d55c9a-7d32-414e-bc29-a96257b19f35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:37.439055 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d55c9a-7d32-414e-bc29-a96257b19f35 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:37.439343 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d55c9a-7d32-414e-bc29-a96257b19f35 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:37.439625 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d55c9a-7d32-414e-bc29-a96257b19f35 None admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:37.440357 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d55c9a-7d32-414e-bc29-a96257b19f35 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:37.440670 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d55c9a-7d32-414e-bc29-a96257b19f35 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:37.442079 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d55c9a-7d32-414e-bc29-a96257b19f35 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:37.455457 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 308/602] 10.222.0.64 () {56 vars in 1109 bytes} [Tue Aug 2 09:46:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:37.456306 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d55c9a-7d32-414e-bc29-a96257b19f35 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:37.481942 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3217eba9-127d-410a-8eac-9a9953de074b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:37.482569 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3217eba9-127d-410a-8eac-9a9953de074b None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:37.482853 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3217eba9-127d-410a-8eac-9a9953de074b None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:37.483126 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3217eba9-127d-410a-8eac-9a9953de074b None admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:37.483851 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3217eba9-127d-410a-8eac-9a9953de074b None admin] RBAC: Adding query filter params (name=service) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:37.484727 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3217eba9-127d-410a-8eac-9a9953de074b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:37.486904 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3217eba9-127d-410a-8eac-9a9953de074b None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:37.497989 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 295/603] 10.222.0.64 () {56 vars in 1103 bytes} [Tue Aug 2 09:46:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:37.498825 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3217eba9-127d-410a-8eac-9a9953de074b None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:37.513141 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:37.532124 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:37.588093 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d55c9a-7d32-414e-bc29-a96257b19f35 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:37.624983 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a7e0e6-b390-40d4-a572-84e7880cf323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:37.625688 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a7e0e6-b390-40d4-a572-84e7880cf323 None admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:37.626485 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a7e0e6-b390-40d4-a572-84e7880cf323 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:37.626812 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a7e0e6-b390-40d4-a572-84e7880cf323 None admin] PATH_INFO: `/v3/projects/390bd7c4f91c4465b0c7ab0f9e436c4d/users/99655e1f9ca24472a86cd26b0dbd1fc2/roles/5704af09562a457fa35364e8af9481f1` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:37.633119 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3217eba9-127d-410a-8eac-9a9953de074b None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:37.643466 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a7e0e6-b390-40d4-a572-84e7880cf323 None admin] RBAC: Authorizing `identity:create_grant(project_id=390bd7c4f91c4465b0c7ab0f9e436c4d, user_id=99655e1f9ca24472a86cd26b0dbd1fc2, role_id=5704af09562a457fa35364e8af9481f1)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:37.644976 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a7e0e6-b390-40d4-a572-84e7880cf323 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:37.659068 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 309/604] 10.222.0.64 () {58 vars in 1439 bytes} [Tue Aug 2 09:46:37 2022] PUT /identity/v3/projects/390bd7c4f91c4465b0c7ab0f9e436c4d/users/99655e1f9ca24472a86cd26b0dbd1fc2/roles/5704af09562a457fa35364e8af9481f1 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:46:37.659896 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a7e0e6-b390-40d4-a572-84e7880cf323 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:37.698091 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488f1101-2ea2-4ef0-9cfe-80ec989e4ea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:37.699488 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488f1101-2ea2-4ef0-9cfe-80ec989e4ea4 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:37.700162 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488f1101-2ea2-4ef0-9cfe-80ec989e4ea4 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:37.700854 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488f1101-2ea2-4ef0-9cfe-80ec989e4ea4 None admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:37.702436 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488f1101-2ea2-4ef0-9cfe-80ec989e4ea4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:37.702948 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488f1101-2ea2-4ef0-9cfe-80ec989e4ea4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:37.705267 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488f1101-2ea2-4ef0-9cfe-80ec989e4ea4 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:37.724760 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 296/605] 10.222.0.64 () {56 vars in 1109 bytes} [Tue Aug 2 09:46:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:37.726292 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488f1101-2ea2-4ef0-9cfe-80ec989e4ea4 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:37.815072 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:37.837683 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a7e0e6-b390-40d4-a572-84e7880cf323 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:37.855530 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:37.875788 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488f1101-2ea2-4ef0-9cfe-80ec989e4ea4 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:37.911755 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bf84c1-2ab4-48cf-90ed-0fe7cfd8e912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:37.912632 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bf84c1-2ab4-48cf-90ed-0fe7cfd8e912 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:37.913186 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bf84c1-2ab4-48cf-90ed-0fe7cfd8e912 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:37.913550 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bf84c1-2ab4-48cf-90ed-0fe7cfd8e912 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:37.920432 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058ff297-9be4-48cd-b7f2-ae0889f508ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:37.921171 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058ff297-9be4-48cd-b7f2-ae0889f508ca None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:37.921498 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bf84c1-2ab4-48cf-90ed-0fe7cfd8e912 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:37.921622 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058ff297-9be4-48cd-b7f2-ae0889f508ca None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:37.921928 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058ff297-9be4-48cd-b7f2-ae0889f508ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:37.923889 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bf84c1-2ab4-48cf-90ed-0fe7cfd8e912 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:37.933859 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058ff297-9be4-48cd-b7f2-ae0889f508ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:37.935450 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058ff297-9be4-48cd-b7f2-ae0889f508ca None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:37.947192 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-66bf84c1-2ab4-48cf-90ed-0fe7cfd8e912 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 09:46:37.947192 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:37.947192 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:37.947192 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:37.947192 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:37.947192 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:37.947192 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:37.947192 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:37.947192 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:37.947192 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:37.947192 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:37.947192 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:37.947192 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 09:46:37.947192 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 09:46:37.947192 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 09:46:37.948432 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 09:46:37.948432 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:37.948432 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:37.948432 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:37.948432 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:37.948432 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:37.948432 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:37.948432 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:37.948432 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:37.948432 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:37.948432 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:37.948432 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:37.948432 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:37.948432 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:37.948432 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:37.948432 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:37.948432 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:37.949583 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 09:46:37.949583 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 09:46:37.949583 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 09:46:37.949583 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 09:46:37.949583 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 09:46:37.949583 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 09:46:37.949583 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 09:46:37.949583 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:46:37.956280 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 310/606] 10.222.0.64 () {56 vars in 1111 bytes} [Tue Aug 2 09:46:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:46:37.956280 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bf84c1-2ab4-48cf-90ed-0fe7cfd8e912 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:37.958081 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-058ff297-9be4-48cd-b7f2-ae0889f508ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:46:37.958081 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:37.958081 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:37.958081 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:37.958081 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:37.958081 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:37.958081 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:37.958081 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:37.958081 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:37.958081 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:37.958081 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:37.958081 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:37.958081 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 09:46:37.958081 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 09:46:37.958081 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 09:46:37.959327 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 09:46:37.959327 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:37.959327 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:37.959327 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:37.959327 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:37.959327 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:37.959327 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:37.959327 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:37.959327 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:37.959327 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:37.959327 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:37.959327 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:37.959327 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:37.959327 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:37.959327 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:37.959327 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:37.959327 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:37.961423 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 09:46:37.961423 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 09:46:37.961423 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:46:37.961423 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:46:37.961423 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 297/607] 10.222.0.64 () {56 vars in 1108 bytes} [Tue Aug 2 09:46:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:46:37.961423 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058ff297-9be4-48cd-b7f2-ae0889f508ca None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:38.067823 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058ff297-9be4-48cd-b7f2-ae0889f508ca None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:38.070231 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bf84c1-2ab4-48cf-90ed-0fe7cfd8e912 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:38.103744 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b5e68d-6259-472b-85f8-1101055d2eab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:38.104402 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b5e68d-6259-472b-85f8-1101055d2eab None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:38.104676 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b5e68d-6259-472b-85f8-1101055d2eab None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:38.104964 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b5e68d-6259-472b-85f8-1101055d2eab None admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:38.105781 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b5e68d-6259-472b-85f8-1101055d2eab None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:38.106041 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b5e68d-6259-472b-85f8-1101055d2eab None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:38.107484 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b5e68d-6259-472b-85f8-1101055d2eab None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:38.109444 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1006d9-1434-4dad-a3e7-2d8fe856551b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:38.110231 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1006d9-1434-4dad-a3e7-2d8fe856551b None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:38.110516 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1006d9-1434-4dad-a3e7-2d8fe856551b None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:38.110823 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1006d9-1434-4dad-a3e7-2d8fe856551b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:38.115125 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:38.119071 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1006d9-1434-4dad-a3e7-2d8fe856551b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:38.121279 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1006d9-1434-4dad-a3e7-2d8fe856551b None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:38.121462 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 298/608] 10.222.0.64 () {56 vars in 1148 bytes} [Tue Aug 2 09:46:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:38.122080 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b5e68d-6259-472b-85f8-1101055d2eab None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:38.135637 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-0e1006d9-1434-4dad-a3e7-2d8fe856551b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 09:46:38.135637 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:38.135637 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:38.135637 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:38.135637 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:38.135637 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:38.135637 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:38.135637 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:38.135637 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:38.135637 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:38.135637 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:38.135637 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:38.135637 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 09:46:38.135637 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 09:46:38.135637 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 09:46:38.137789 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 09:46:38.137789 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:38.137789 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:38.137789 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:38.137789 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:38.137789 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:38.137789 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:38.137789 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:38.137789 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:38.137789 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:38.137789 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:38.137789 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:38.137789 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:38.137789 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:38.137789 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:38.137789 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:38.137789 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:38.139872 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 09:46:38.139872 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 09:46:38.139872 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 09:46:38.139872 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 09:46:38.139872 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 09:46:38.139872 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 09:46:38.139872 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 09:46:38.139872 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:46:38.139872 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 311/609] 10.222.0.64 () {56 vars in 1111 bytes} [Tue Aug 2 09:46:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:46:38.139872 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1006d9-1434-4dad-a3e7-2d8fe856551b None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:38.150210 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:38.234593 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b5e68d-6259-472b-85f8-1101055d2eab None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:38.255760 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1006d9-1434-4dad-a3e7-2d8fe856551b None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:38.271801 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d2897c-5820-4890-979e-06348fbc65d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:38.272268 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d2897c-5820-4890-979e-06348fbc65d2 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:38.272555 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d2897c-5820-4890-979e-06348fbc65d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:38.272841 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d2897c-5820-4890-979e-06348fbc65d2 None admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:38.273550 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d2897c-5820-4890-979e-06348fbc65d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:38.273944 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d2897c-5820-4890-979e-06348fbc65d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:38.275223 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d2897c-5820-4890-979e-06348fbc65d2 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:38.287965 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 299/610] 10.222.0.64 () {56 vars in 1109 bytes} [Tue Aug 2 09:46:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:38.288961 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d2897c-5820-4890-979e-06348fbc65d2 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:38.295084 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318410eb-606b-4327-9cfb-f36a2d3bc543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:38.296084 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318410eb-606b-4327-9cfb-f36a2d3bc543 None admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:38.296767 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318410eb-606b-4327-9cfb-f36a2d3bc543 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:38.297278 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318410eb-606b-4327-9cfb-f36a2d3bc543 None admin] PATH_INFO: `/v3/projects/390bd7c4f91c4465b0c7ab0f9e436c4d/users/04fb1dbe260a4721aaa0bd0493260d33/roles/5704af09562a457fa35364e8af9481f1` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:38.314829 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318410eb-606b-4327-9cfb-f36a2d3bc543 None admin] RBAC: Authorizing `identity:create_grant(project_id=390bd7c4f91c4465b0c7ab0f9e436c4d, user_id=04fb1dbe260a4721aaa0bd0493260d33, role_id=5704af09562a457fa35364e8af9481f1)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:38.316675 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318410eb-606b-4327-9cfb-f36a2d3bc543 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:38.335751 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 312/611] 10.222.0.64 () {58 vars in 1439 bytes} [Tue Aug 2 09:46:38 2022] PUT /identity/v3/projects/390bd7c4f91c4465b0c7ab0f9e436c4d/users/04fb1dbe260a4721aaa0bd0493260d33/roles/5704af09562a457fa35364e8af9481f1 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:46:38.337131 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318410eb-606b-4327-9cfb-f36a2d3bc543 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:38.379192 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:38.465407 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d2897c-5820-4890-979e-06348fbc65d2 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:38.482275 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:38.549035 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ccf98f-aafa-42ac-9b43-3587bb986865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:38.549677 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ccf98f-aafa-42ac-9b43-3587bb986865 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:38.549962 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ccf98f-aafa-42ac-9b43-3587bb986865 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:38.550239 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ccf98f-aafa-42ac-9b43-3587bb986865 None admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:38.551025 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ccf98f-aafa-42ac-9b43-3587bb986865 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:38.551334 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ccf98f-aafa-42ac-9b43-3587bb986865 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:38.552854 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ccf98f-aafa-42ac-9b43-3587bb986865 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:38.575421 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318410eb-606b-4327-9cfb-f36a2d3bc543 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:38.584438 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 300/612] 10.222.0.64 () {56 vars in 1148 bytes} [Tue Aug 2 09:46:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:38.588092 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ccf98f-aafa-42ac-9b43-3587bb986865 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:38.637716 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33da7c7-c4fb-4c69-a611-8b85ee27944f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:38.638176 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33da7c7-c4fb-4c69-a611-8b85ee27944f None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:38.638468 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33da7c7-c4fb-4c69-a611-8b85ee27944f None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:38.638742 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33da7c7-c4fb-4c69-a611-8b85ee27944f None admin] PATH_INFO: `/v3/users/swift` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:38.650980 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33da7c7-c4fb-4c69-a611-8b85ee27944f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:38.653148 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33da7c7-c4fb-4c69-a611-8b85ee27944f None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:38.670530 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-d33da7c7-c4fb-4c69-a611-8b85ee27944f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 02 09:46:38.670530 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:38.670530 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:38.670530 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:38.670530 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:38.670530 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:38.670530 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:38.670530 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:38.670530 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:38.670530 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:38.670530 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:38.670530 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:38.670530 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 09:46:38.670530 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 09:46:38.670530 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 09:46:38.670530 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 09:46:38.671587 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:38.671587 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:38.671587 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 09:46:38.671587 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 09:46:38.671587 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 09:46:38.671587 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 09:46:38.671587 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:38.671587 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:38.671587 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:38.671587 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:38.671587 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:38.671587 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:38.671587 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:38.671587 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:38.671587 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:38.671587 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:38.671587 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:38.671587 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:38.672658 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:38.672658 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:38.672658 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 09:46:38.672658 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 09:46:38.672658 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 09:46:38.672658 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 09:46:38.672658 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 09:46:38.672658 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 09:46:38.672658 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 02 09:46:38.672658 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:46:38.672658 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 313/613] 10.222.0.64 () {56 vars in 1096 bytes} [Tue Aug 2 09:46:38 2022] GET /identity/v3/users/swift => generated 83 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:46:38.677339 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33da7c7-c4fb-4c69-a611-8b85ee27944f None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:38.723990 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ccf98f-aafa-42ac-9b43-3587bb986865 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:38.743537 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:38.762979 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4cd768-156d-47e5-8d2e-d2b4098c6943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:38.763940 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4cd768-156d-47e5-8d2e-d2b4098c6943 None admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:38.764418 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4cd768-156d-47e5-8d2e-d2b4098c6943 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:38.764900 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4cd768-156d-47e5-8d2e-d2b4098c6943 None admin] PATH_INFO: `/v3/projects/390bd7c4f91c4465b0c7ab0f9e436c4d/users/ec31b188970947efb2ed1d4d9a31cadf/roles/5704af09562a457fa35364e8af9481f1` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:38.781559 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4cd768-156d-47e5-8d2e-d2b4098c6943 None admin] RBAC: Authorizing `identity:create_grant(project_id=390bd7c4f91c4465b0c7ab0f9e436c4d, user_id=ec31b188970947efb2ed1d4d9a31cadf, role_id=5704af09562a457fa35364e8af9481f1)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:38.783433 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4cd768-156d-47e5-8d2e-d2b4098c6943 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:38.801597 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 301/614] 10.222.0.64 () {58 vars in 1439 bytes} [Tue Aug 2 09:46:38 2022] PUT /identity/v3/projects/390bd7c4f91c4465b0c7ab0f9e436c4d/users/ec31b188970947efb2ed1d4d9a31cadf/roles/5704af09562a457fa35364e8af9481f1 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:46:38.808156 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a76902-623a-4528-a69a-ed76651edd23 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:38.808702 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a76902-623a-4528-a69a-ed76651edd23 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:38.809127 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a76902-623a-4528-a69a-ed76651edd23 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:38.810741 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 302/615] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:46:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:46:38.822389 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ab2152-70a4-49d3-8b23-cb4fc05872e7 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:38.822997 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ab2152-70a4-49d3-8b23-cb4fc05872e7 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:38.823423 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ab2152-70a4-49d3-8b23-cb4fc05872e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:38.859069 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-e5ab2152-70a4-49d3-8b23-cb4fc05872e7 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:46:38.862105 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-e5ab2152-70a4-49d3-8b23-cb4fc05872e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:46:38.872168 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33da7c7-c4fb-4c69-a611-8b85ee27944f None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:38.916127 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:38.957665 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376db477-b753-4281-a832-ab92d20fc5ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:38.958561 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376db477-b753-4281-a832-ab92d20fc5ad None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:38.959455 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376db477-b753-4281-a832-ab92d20fc5ad None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:38.959903 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376db477-b753-4281-a832-ab92d20fc5ad None admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:38.962289 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376db477-b753-4281-a832-ab92d20fc5ad None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:38.962801 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376db477-b753-4281-a832-ab92d20fc5ad None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:38.965392 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376db477-b753-4281-a832-ab92d20fc5ad None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:38.989439 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 303/616] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:46:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 09:46:38.996409 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 314/617] 10.222.0.64 () {56 vars in 1135 bytes} [Tue Aug 2 09:46:38 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:46:39.001537 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ab2152-70a4-49d3-8b23-cb4fc05872e7 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:39.021143 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376db477-b753-4281-a832-ab92d20fc5ad None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:39.132278 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:39.138919 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ab2152-70a4-49d3-8b23-cb4fc05872e7 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:39.191173 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376db477-b753-4281-a832-ab92d20fc5ad None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:39.207032 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf65a18d-c4f7-4b3a-9898-f8d0b82a1944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:39.208000 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf65a18d-c4f7-4b3a-9898-f8d0b82a1944 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:39.208550 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf65a18d-c4f7-4b3a-9898-f8d0b82a1944 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:39.208977 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf65a18d-c4f7-4b3a-9898-f8d0b82a1944 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:39.228276 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf65a18d-c4f7-4b3a-9898-f8d0b82a1944 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:39.230169 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf65a18d-c4f7-4b3a-9898-f8d0b82a1944 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:39.230351 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd6a0b8-98ec-40f4-ad37-be9d74c94b85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:39.236271 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd6a0b8-98ec-40f4-ad37-be9d74c94b85 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:39.236271 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd6a0b8-98ec-40f4-ad37-be9d74c94b85 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:39.236271 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd6a0b8-98ec-40f4-ad37-be9d74c94b85 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:39.246437 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd6a0b8-98ec-40f4-ad37-be9d74c94b85 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 09:46:39.252529 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-3dd6a0b8-98ec-40f4-ad37-be9d74c94b85 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 09:46:39.252529 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:39.252529 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:39.252529 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:39.252529 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:39.252529 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:39.252529 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:39.252529 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:39.252529 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:39.252529 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:39.252529 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:39.252529 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:39.252529 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 09:46:39.252529 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 09:46:39.252529 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 09:46:39.252529 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 09:46:39.253534 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 09:46:39.253534 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 09:46:39.253534 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 09:46:39.253534 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 09:46:39.253534 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:39.253534 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:39.253534 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:39.253534 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:39.253534 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:39.253534 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:39.253534 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:39.253534 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:39.253534 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:39.253534 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:39.253534 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:39.253534 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:39.254604 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:39.254604 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:39.254604 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:39.254604 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:39.254604 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 09:46:39.254604 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 09:46:39.254604 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 09:46:39.254604 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 09:46:39.254604 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 09:46:39.254604 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 09:46:39.254604 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 09:46:39.254604 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:46:39.254604 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 315/618] 10.222.0.64 () {56 vars in 1102 bytes} [Tue Aug 2 09:46:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 09:46:39.260105 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd6a0b8-98ec-40f4-ad37-be9d74c94b85 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:39.269009 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-bf65a18d-c4f7-4b3a-9898-f8d0b82a1944 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:46:39.269009 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:39.269009 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:39.269009 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:39.269009 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:39.269009 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:39.269009 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:39.269009 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:39.269009 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:39.269009 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:39.269009 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:39.269009 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:39.269009 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 09:46:39.269009 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 09:46:39.269009 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 09:46:39.270282 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 09:46:39.270282 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:39.270282 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:39.270282 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:39.270282 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:39.270282 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:39.270282 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:39.270282 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:39.270282 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:39.270282 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:39.270282 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:39.270282 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:39.270282 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:39.270282 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:39.270282 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:39.270282 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:39.270282 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:39.271387 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 09:46:39.271387 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 09:46:39.271387 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:46:39.271387 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:46:39.274774 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 304/619] 10.222.0.64 () {56 vars in 1108 bytes} [Tue Aug 2 09:46:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:46:39.281565 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf65a18d-c4f7-4b3a-9898-f8d0b82a1944 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:39.323625 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:39.455657 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf65a18d-c4f7-4b3a-9898-f8d0b82a1944 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:39.465230 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd6a0b8-98ec-40f4-ad37-be9d74c94b85 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:39.493994 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b664e5-a568-4d58-9b25-4da12953870d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:39.494735 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b664e5-a568-4d58-9b25-4da12953870d None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:39.495051 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b664e5-a568-4d58-9b25-4da12953870d None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:39.495244 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b664e5-a568-4d58-9b25-4da12953870d None admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:39.495885 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b664e5-a568-4d58-9b25-4da12953870d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:39.496193 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b664e5-a568-4d58-9b25-4da12953870d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:39.497745 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b664e5-a568-4d58-9b25-4da12953870d None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:39.511722 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 305/620] 10.222.0.64 () {56 vars in 1109 bytes} [Tue Aug 2 09:46:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:46:39.516757 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b664e5-a568-4d58-9b25-4da12953870d None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:39.530931 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:39.552959 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a4a7ff-0db1-44fd-b61b-1bfc1d57347a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:39.553643 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a4a7ff-0db1-44fd-b61b-1bfc1d57347a None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:39.553866 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a4a7ff-0db1-44fd-b61b-1bfc1d57347a None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:39.554140 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a4a7ff-0db1-44fd-b61b-1bfc1d57347a None admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:39.554874 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a4a7ff-0db1-44fd-b61b-1bfc1d57347a None admin] RBAC: Adding query filter params (name=service) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:39.555180 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a4a7ff-0db1-44fd-b61b-1bfc1d57347a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:39.556594 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a4a7ff-0db1-44fd-b61b-1bfc1d57347a None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:39.582527 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 316/621] 10.222.0.64 () {56 vars in 1103 bytes} [Tue Aug 2 09:46:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:39.589319 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a4a7ff-0db1-44fd-b61b-1bfc1d57347a None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:39.627843 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b664e5-a568-4d58-9b25-4da12953870d None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:39.665927 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba8123d-d5f1-4b21-b3d5-1c92d8e8d903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:39.666585 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba8123d-d5f1-4b21-b3d5-1c92d8e8d903 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:39.666876 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba8123d-d5f1-4b21-b3d5-1c92d8e8d903 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:39.667154 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba8123d-d5f1-4b21-b3d5-1c92d8e8d903 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:39.675276 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba8123d-d5f1-4b21-b3d5-1c92d8e8d903 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:39.676786 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba8123d-d5f1-4b21-b3d5-1c92d8e8d903 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:39.689896 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-0ba8123d-d5f1-4b21-b3d5-1c92d8e8d903 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 09:46:39.689896 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:39.689896 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:39.689896 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:39.689896 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:39.689896 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:39.689896 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:39.689896 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:39.689896 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:39.689896 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:39.689896 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:39.689896 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:39.689896 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 09:46:39.689896 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 09:46:39.689896 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 09:46:39.690972 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 09:46:39.690972 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:39.690972 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:39.690972 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:39.690972 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:39.690972 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:39.690972 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:39.690972 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:39.690972 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:39.690972 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:39.690972 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:39.690972 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:39.690972 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:39.690972 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:39.690972 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:39.690972 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:39.690972 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:39.692499 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 09:46:39.692499 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 09:46:39.692499 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 09:46:39.692499 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 09:46:39.692499 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 09:46:39.692499 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 09:46:39.692499 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 09:46:39.692499 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:46:39.692499 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 306/622] 10.222.0.64 () {56 vars in 1111 bytes} [Tue Aug 2 09:46:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:46:39.696759 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba8123d-d5f1-4b21-b3d5-1c92d8e8d903 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:39.800687 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:39.831209 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:39.837088 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a4a7ff-0db1-44fd-b61b-1bfc1d57347a None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:39.838410 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba8123d-d5f1-4b21-b3d5-1c92d8e8d903 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:39.879763 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcfa03d-80ef-44f3-9260-4025f06a5f95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:39.880434 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcfa03d-80ef-44f3-9260-4025f06a5f95 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:39.880720 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcfa03d-80ef-44f3-9260-4025f06a5f95 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:39.881004 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcfa03d-80ef-44f3-9260-4025f06a5f95 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:39.905733 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcfa03d-80ef-44f3-9260-4025f06a5f95 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:39.907783 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcfa03d-80ef-44f3-9260-4025f06a5f95 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:39.921148 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff835dc-2009-4b5b-950a-d80fc0965ea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:39.921777 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff835dc-2009-4b5b-950a-d80fc0965ea9 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:39.922061 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff835dc-2009-4b5b-950a-d80fc0965ea9 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:39.922335 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff835dc-2009-4b5b-950a-d80fc0965ea9 None admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:39.923812 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff835dc-2009-4b5b-950a-d80fc0965ea9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:39.924402 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff835dc-2009-4b5b-950a-d80fc0965ea9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:39.929577 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff835dc-2009-4b5b-950a-d80fc0965ea9 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:39.936895 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-fbcfa03d-80ef-44f3-9260-4025f06a5f95 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:46:39.936895 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:39.936895 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:39.936895 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:39.936895 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:39.936895 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:39.936895 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:39.936895 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:39.936895 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:39.936895 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:39.936895 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:39.936895 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:39.936895 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 09:46:39.936895 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 09:46:39.936895 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 09:46:39.937828 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 09:46:39.937828 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:39.937828 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:39.937828 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:39.937828 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:39.937828 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:39.937828 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:39.937828 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:39.937828 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:39.937828 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:39.937828 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:39.937828 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:39.937828 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:39.937828 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:39.937828 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:39.937828 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:39.937828 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:39.939340 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 09:46:39.939340 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 09:46:39.939340 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:46:39.939340 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:46:39.939340 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 317/623] 10.222.0.64 () {56 vars in 1108 bytes} [Tue Aug 2 09:46:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:46:39.943801 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcfa03d-80ef-44f3-9260-4025f06a5f95 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:39.959201 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 307/624] 10.222.0.64 () {56 vars in 1148 bytes} [Tue Aug 2 09:46:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:46:39.965231 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff835dc-2009-4b5b-950a-d80fc0965ea9 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:40.129483 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcfa03d-80ef-44f3-9260-4025f06a5f95 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:40.137869 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff835dc-2009-4b5b-950a-d80fc0965ea9 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:40.165480 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d811e6-d80c-441b-ab7b-2c7109bac65b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:40.166371 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d811e6-d80c-441b-ab7b-2c7109bac65b None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:40.166653 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d811e6-d80c-441b-ab7b-2c7109bac65b None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:40.167352 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d811e6-d80c-441b-ab7b-2c7109bac65b None admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:40.168414 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d811e6-d80c-441b-ab7b-2c7109bac65b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:40.168726 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d811e6-d80c-441b-ab7b-2c7109bac65b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:40.170924 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d811e6-d80c-441b-ab7b-2c7109bac65b None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:40.172419 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:40.179981 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb80586c-9268-4c5b-8c84-3001969a08e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:40.180679 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb80586c-9268-4c5b-8c84-3001969a08e0 None admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:40.180965 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb80586c-9268-4c5b-8c84-3001969a08e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:40.181242 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb80586c-9268-4c5b-8c84-3001969a08e0 None admin] PATH_INFO: `/v3/projects/390bd7c4f91c4465b0c7ab0f9e436c4d/users/6a371e3df3764a4cbb1b8368aa74bd65/roles/5704af09562a457fa35364e8af9481f1` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:40.193507 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 318/625] 10.222.0.64 () {56 vars in 1109 bytes} [Tue Aug 2 09:46:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:46:40.193709 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd7c15d-a332-4636-a3d4-5f5e41b08c2c None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:40.194329 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd7c15d-a332-4636-a3d4-5f5e41b08c2c None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:40.194329 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd7c15d-a332-4636-a3d4-5f5e41b08c2c None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:40.197225 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 319/626] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:46:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:46:40.198422 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd7c15d-a332-4636-a3d4-5f5e41b08c2c None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:40.205516 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb80586c-9268-4c5b-8c84-3001969a08e0 None admin] RBAC: Authorizing `identity:create_grant(project_id=390bd7c4f91c4465b0c7ab0f9e436c4d, user_id=6a371e3df3764a4cbb1b8368aa74bd65, role_id=5704af09562a457fa35364e8af9481f1)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:40.207168 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb80586c-9268-4c5b-8c84-3001969a08e0 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:40.218691 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:40.225876 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 308/627] 10.222.0.64 () {58 vars in 1439 bytes} [Tue Aug 2 09:46:39 2022] PUT /identity/v3/projects/390bd7c4f91c4465b0c7ab0f9e436c4d/users/6a371e3df3764a4cbb1b8368aa74bd65/roles/5704af09562a457fa35364e8af9481f1 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:46:40.227389 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d49e05d-9196-48f4-bdfd-306619329c82 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:40.227686 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d49e05d-9196-48f4-bdfd-306619329c82 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:40.227966 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d49e05d-9196-48f4-bdfd-306619329c82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:40.251972 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-6d49e05d-9196-48f4-bdfd-306619329c82 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:46:40.253243 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-6d49e05d-9196-48f4-bdfd-306619329c82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:46:40.340013 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd7c15d-a332-4636-a3d4-5f5e41b08c2c None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:40.341668 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 309/628] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:46:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 09:46:40.367960 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d49e05d-9196-48f4-bdfd-306619329c82 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:40.389634 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f262770-3023-4ff5-ad3d-638c710376c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:40.390516 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f262770-3023-4ff5-ad3d-638c710376c1 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:40.391004 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f262770-3023-4ff5-ad3d-638c710376c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:40.392050 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f262770-3023-4ff5-ad3d-638c710376c1 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:40.403936 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f262770-3023-4ff5-ad3d-638c710376c1 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:40.406215 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f262770-3023-4ff5-ad3d-638c710376c1 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:40.423823 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-3f262770-3023-4ff5-ad3d-638c710376c1 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 02 09:46:40.423823 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:40.423823 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:40.423823 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:40.423823 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:40.423823 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:40.423823 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:40.423823 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:40.423823 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:40.423823 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:40.423823 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:40.423823 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:40.423823 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 09:46:40.423823 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 09:46:40.423823 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 09:46:40.423823 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 09:46:40.424986 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:40.424986 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:40.424986 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 09:46:40.424986 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 09:46:40.424986 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 09:46:40.424986 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 09:46:40.424986 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:40.424986 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:40.424986 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:40.424986 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:40.424986 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:40.424986 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:40.424986 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:40.424986 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:40.424986 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:40.424986 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:40.424986 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:40.424986 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:40.425954 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:40.425954 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:40.425954 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 09:46:40.425954 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 09:46:40.425954 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 09:46:40.425954 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 09:46:40.425954 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 09:46:40.425954 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 09:46:40.425954 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 02 09:46:40.425954 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:46:40.425954 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 320/629] 10.222.0.64 () {56 vars in 1099 bytes} [Tue Aug 2 09:46:40 2022] GET /identity/v3/users/cinder => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:46:40.432651 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f262770-3023-4ff5-ad3d-638c710376c1 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:40.491686 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d49e05d-9196-48f4-bdfd-306619329c82 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:40.505397 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:40.522165 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:40.541142 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3275ba7a-54b3-4cae-a59c-94890e2ac5c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:40.541857 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3275ba7a-54b3-4cae-a59c-94890e2ac5c2 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:40.542148 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3275ba7a-54b3-4cae-a59c-94890e2ac5c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:40.542897 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3275ba7a-54b3-4cae-a59c-94890e2ac5c2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:40.557804 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3275ba7a-54b3-4cae-a59c-94890e2ac5c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 09:46:40.562893 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f262770-3023-4ff5-ad3d-638c710376c1 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:40.564004 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-3275ba7a-54b3-4cae-a59c-94890e2ac5c2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 09:46:40.564004 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:40.564004 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:40.564004 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:40.564004 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:40.564004 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:40.564004 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:40.564004 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:40.564004 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:40.564004 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:40.564004 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:40.564004 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:40.564004 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 09:46:40.564004 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 09:46:40.564004 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 09:46:40.564004 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 09:46:40.565178 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 09:46:40.565178 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 09:46:40.565178 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 09:46:40.565178 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 09:46:40.565178 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:40.565178 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:40.565178 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:40.565178 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:40.565178 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:40.565178 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:40.565178 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:40.565178 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:40.565178 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:40.565178 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:40.565178 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:40.565178 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:40.566225 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:40.566225 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:40.566225 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:40.566225 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:40.566225 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 09:46:40.566225 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 09:46:40.566225 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 09:46:40.566225 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 09:46:40.566225 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 09:46:40.566225 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 09:46:40.566225 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 09:46:40.566225 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:46:40.566225 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 310/630] 10.222.0.64 () {56 vars in 1102 bytes} [Tue Aug 2 09:46:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 09:46:40.572215 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3275ba7a-54b3-4cae-a59c-94890e2ac5c2 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:40.653073 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55894971-15c7-44bb-9851-172251e80a88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:40.653982 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55894971-15c7-44bb-9851-172251e80a88 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:40.654420 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55894971-15c7-44bb-9851-172251e80a88 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:40.654916 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55894971-15c7-44bb-9851-172251e80a88 None admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:40.655876 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55894971-15c7-44bb-9851-172251e80a88 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:40.656367 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55894971-15c7-44bb-9851-172251e80a88 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:40.658618 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55894971-15c7-44bb-9851-172251e80a88 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:40.688552 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3275ba7a-54b3-4cae-a59c-94890e2ac5c2 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:40.702629 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 321/631] 10.222.0.64 () {56 vars in 1137 bytes} [Tue Aug 2 09:46:40 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:40.713329 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55894971-15c7-44bb-9851-172251e80a88 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:40.726856 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bac1e7-b702-4aef-884a-f1b10a9aa3f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:40.727476 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bac1e7-b702-4aef-884a-f1b10a9aa3f6 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:40.727759 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bac1e7-b702-4aef-884a-f1b10a9aa3f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:40.728034 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bac1e7-b702-4aef-884a-f1b10a9aa3f6 None admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:40.728788 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bac1e7-b702-4aef-884a-f1b10a9aa3f6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:40.729092 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bac1e7-b702-4aef-884a-f1b10a9aa3f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:40.735448 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bac1e7-b702-4aef-884a-f1b10a9aa3f6 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:40.741441 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 311/632] 10.222.0.64 () {56 vars in 1103 bytes} [Tue Aug 2 09:46:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:40.746744 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bac1e7-b702-4aef-884a-f1b10a9aa3f6 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:40.796740 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:40.867802 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bac1e7-b702-4aef-884a-f1b10a9aa3f6 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:40.905269 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277f182e-a0fc-44b3-a205-7bb4863b70c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:40.905926 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277f182e-a0fc-44b3-a205-7bb4863b70c1 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:40.906276 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277f182e-a0fc-44b3-a205-7bb4863b70c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:40.906617 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277f182e-a0fc-44b3-a205-7bb4863b70c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:40.918844 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277f182e-a0fc-44b3-a205-7bb4863b70c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:40.920408 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277f182e-a0fc-44b3-a205-7bb4863b70c1 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:40.936863 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-277f182e-a0fc-44b3-a205-7bb4863b70c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:46:40.936863 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:40.936863 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:40.936863 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:40.936863 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:40.936863 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:40.936863 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:40.936863 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:40.936863 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:40.936863 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:40.936863 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:40.936863 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:40.936863 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 09:46:40.936863 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 09:46:40.936863 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 09:46:40.937851 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 09:46:40.937851 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:40.937851 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:40.937851 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:40.937851 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:40.937851 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:40.937851 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:40.937851 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:40.937851 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:40.937851 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:40.937851 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:40.937851 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:40.937851 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:40.937851 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:40.937851 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:40.937851 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:40.937851 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:40.939310 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 09:46:40.939310 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 09:46:40.939310 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:46:40.939310 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:46:40.940873 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 312/633] 10.222.0.64 () {56 vars in 1108 bytes} [Tue Aug 2 09:46:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 09:46:40.945434 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277f182e-a0fc-44b3-a205-7bb4863b70c1 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:41.040790 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:41.060934 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277f182e-a0fc-44b3-a205-7bb4863b70c1 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:41.076700 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55894971-15c7-44bb-9851-172251e80a88 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:41.118256 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:41.128588 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660bc65f-7066-41c7-baf0-1dd6a5c62f9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:41.129673 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660bc65f-7066-41c7-baf0-1dd6a5c62f9a None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:41.130106 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660bc65f-7066-41c7-baf0-1dd6a5c62f9a None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:41.130684 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660bc65f-7066-41c7-baf0-1dd6a5c62f9a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:41.136278 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a70b2a-d92d-48bb-9847-a7533c851be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:41.137034 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a70b2a-d92d-48bb-9847-a7533c851be8 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:41.137416 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a70b2a-d92d-48bb-9847-a7533c851be8 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:41.137792 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a70b2a-d92d-48bb-9847-a7533c851be8 None admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:41.138598 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a70b2a-d92d-48bb-9847-a7533c851be8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:41.139008 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a70b2a-d92d-48bb-9847-a7533c851be8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:41.140530 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a70b2a-d92d-48bb-9847-a7533c851be8 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:41.153297 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 313/634] 10.222.0.64 () {56 vars in 1109 bytes} [Tue Aug 2 09:46:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:46:41.158966 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3250af75-9a05-4a7a-ba66-b188fbf7b7c8 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:41.158966 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3250af75-9a05-4a7a-ba66-b188fbf7b7c8 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:41.158966 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3250af75-9a05-4a7a-ba66-b188fbf7b7c8 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:41.158966 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 314/635] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:46:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:46:41.160563 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3250af75-9a05-4a7a-ba66-b188fbf7b7c8 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:41.220143 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660bc65f-7066-41c7-baf0-1dd6a5c62f9a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:41.222173 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660bc65f-7066-41c7-baf0-1dd6a5c62f9a None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:41.302655 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3250af75-9a05-4a7a-ba66-b188fbf7b7c8 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:41.342765 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 322/636] 10.222.0.64 () {58 vars in 1282 bytes} [Tue Aug 2 09:46:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 630 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 09:46:41.344898 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca25f7c6-7640-4db2-b3e4-98fbaf860bf6 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:41.345383 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca25f7c6-7640-4db2-b3e4-98fbaf860bf6 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:41.345788 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca25f7c6-7640-4db2-b3e4-98fbaf860bf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:41.348385 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df7f257-5825-4669-b3e6-13928b1eef4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:41.349577 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df7f257-5825-4669-b3e6-13928b1eef4e None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:41.351003 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df7f257-5825-4669-b3e6-13928b1eef4e None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:41.351389 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df7f257-5825-4669-b3e6-13928b1eef4e None admin] PATH_INFO: `/v3/users/nova` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:41.368063 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df7f257-5825-4669-b3e6-13928b1eef4e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:41.368063 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df7f257-5825-4669-b3e6-13928b1eef4e None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:41.376638 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-ca25f7c6-7640-4db2-b3e4-98fbaf860bf6 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:46:41.377950 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-ca25f7c6-7640-4db2-b3e4-98fbaf860bf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:46:41.380942 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-0df7f257-5825-4669-b3e6-13928b1eef4e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 02 09:46:41.380942 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:41.380942 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:41.380942 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:41.380942 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:41.380942 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:41.380942 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:41.380942 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:41.380942 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:41.380942 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:41.380942 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:41.380942 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:41.380942 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 09:46:41.380942 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 09:46:41.380942 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 09:46:41.380942 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 09:46:41.381934 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:41.381934 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:41.381934 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 09:46:41.381934 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 09:46:41.381934 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 09:46:41.381934 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 09:46:41.381934 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:41.381934 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:41.381934 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:41.381934 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:41.381934 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:41.381934 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:41.381934 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:41.381934 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:41.381934 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:41.381934 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:41.381934 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:41.381934 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:41.382908 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:41.382908 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:41.382908 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 09:46:41.382908 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 09:46:41.382908 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 09:46:41.382908 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 09:46:41.382908 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 09:46:41.382908 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 09:46:41.382908 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 02 09:46:41.382908 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:46:41.386863 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 315/637] 10.222.0.64 () {56 vars in 1093 bytes} [Tue Aug 2 09:46:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:46:41.387591 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df7f257-5825-4669-b3e6-13928b1eef4e None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:41.391340 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:41.402506 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:41.456226 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 323/638] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:46:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 09:46:41.457451 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844a3362-31dd-417d-8f27-e97cf50f55e7 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:41.457731 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844a3362-31dd-417d-8f27-e97cf50f55e7 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:41.458121 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844a3362-31dd-417d-8f27-e97cf50f55e7 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:41.463319 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 324/639] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:46:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:46:41.463319 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844a3362-31dd-417d-8f27-e97cf50f55e7 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:41.499650 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df7f257-5825-4669-b3e6-13928b1eef4e None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:41.535229 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008a4e2d-1970-4188-b214-3a537d583038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:41.535879 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008a4e2d-1970-4188-b214-3a537d583038 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:41.536156 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008a4e2d-1970-4188-b214-3a537d583038 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:41.536468 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008a4e2d-1970-4188-b214-3a537d583038 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:41.548131 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008a4e2d-1970-4188-b214-3a537d583038 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:41.549812 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008a4e2d-1970-4188-b214-3a537d583038 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:41.565447 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-008a4e2d-1970-4188-b214-3a537d583038 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:46:41.565447 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:41.565447 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:41.565447 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:41.565447 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:41.565447 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:41.565447 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:41.565447 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:41.565447 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:41.565447 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:41.565447 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:41.565447 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:41.565447 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 09:46:41.565447 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 09:46:41.565447 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 09:46:41.566398 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 09:46:41.566398 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:41.566398 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:41.566398 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:41.566398 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:41.566398 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:41.566398 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:41.566398 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:41.566398 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:41.566398 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:41.566398 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:41.566398 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:41.566398 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:41.566398 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:41.566398 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:41.566398 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:41.566398 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:41.567323 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 09:46:41.567323 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 09:46:41.567323 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:46:41.567323 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:46:41.568699 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 316/640] 10.222.0.64 () {56 vars in 1108 bytes} [Tue Aug 2 09:46:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:46:41.570220 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d043bc-65dc-4786-81dd-9b297c15439b None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:41.570893 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d043bc-65dc-4786-81dd-9b297c15439b None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:41.571283 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d043bc-65dc-4786-81dd-9b297c15439b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:41.589410 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-d3d043bc-65dc-4786-81dd-9b297c15439b None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:46:41.590601 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-d3d043bc-65dc-4786-81dd-9b297c15439b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:46:41.591508 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844a3362-31dd-417d-8f27-e97cf50f55e7 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:41.629504 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92351e3e-2f5e-4e7d-a60c-1b74b1a909ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:41.630122 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92351e3e-2f5e-4e7d-a60c-1b74b1a909ef None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:41.630430 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92351e3e-2f5e-4e7d-a60c-1b74b1a909ef None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:41.630678 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92351e3e-2f5e-4e7d-a60c-1b74b1a909ef None admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:41.631594 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92351e3e-2f5e-4e7d-a60c-1b74b1a909ef None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:41.631811 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92351e3e-2f5e-4e7d-a60c-1b74b1a909ef None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:41.633351 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92351e3e-2f5e-4e7d-a60c-1b74b1a909ef None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:41.654121 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 325/641] 10.222.0.64 () {56 vars in 1133 bytes} [Tue Aug 2 09:46:41 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:41.654972 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92351e3e-2f5e-4e7d-a60c-1b74b1a909ef None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:41.659826 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:41.679655 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 317/642] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:46:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 09:46:41.680535 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d043bc-65dc-4786-81dd-9b297c15439b None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:41.692981 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:41.827999 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92351e3e-2f5e-4e7d-a60c-1b74b1a909ef None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:41.832614 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d043bc-65dc-4786-81dd-9b297c15439b None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:41.871129 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed883be-dd05-4078-ad2e-72f0bf0d2d77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:41.871764 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed883be-dd05-4078-ad2e-72f0bf0d2d77 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:41.872043 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed883be-dd05-4078-ad2e-72f0bf0d2d77 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:41.872382 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed883be-dd05-4078-ad2e-72f0bf0d2d77 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:41.881037 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6ac660-138c-48c5-9930-4731699636fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:41.881660 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6ac660-138c-48c5-9930-4731699636fd None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:41.881944 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6ac660-138c-48c5-9930-4731699636fd None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:41.882223 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6ac660-138c-48c5-9930-4731699636fd None admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:41.882921 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6ac660-138c-48c5-9930-4731699636fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:41.883226 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6ac660-138c-48c5-9930-4731699636fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:41.884333 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed883be-dd05-4078-ad2e-72f0bf0d2d77 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 09:46:41.884627 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6ac660-138c-48c5-9930-4731699636fd None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:41.889214 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-3ed883be-dd05-4078-ad2e-72f0bf0d2d77 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 09:46:41.889214 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:41.889214 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:41.889214 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:41.889214 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:41.889214 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:41.889214 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:41.889214 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:41.889214 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:41.889214 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:41.889214 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:41.889214 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:41.889214 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 09:46:41.889214 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 09:46:41.889214 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 09:46:41.889214 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 09:46:41.890220 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 09:46:41.890220 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 09:46:41.890220 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 09:46:41.890220 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 09:46:41.890220 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:41.890220 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:41.890220 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:41.890220 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:41.890220 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:41.890220 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:41.890220 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:41.890220 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:41.890220 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:41.890220 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:41.890220 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:41.890220 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:41.891090 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:41.891090 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:41.891090 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:41.891090 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:41.891090 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 09:46:41.891090 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 09:46:41.891090 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 09:46:41.891090 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 09:46:41.891090 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 09:46:41.891090 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 09:46:41.891090 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 09:46:41.891090 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:46:41.891090 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 326/643] 10.222.0.64 () {56 vars in 1102 bytes} [Tue Aug 2 09:46:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:46:41.893527 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed883be-dd05-4078-ad2e-72f0bf0d2d77 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:41.907128 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 318/644] 10.222.0.64 () {56 vars in 1109 bytes} [Tue Aug 2 09:46:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:41.908980 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6ac660-138c-48c5-9930-4731699636fd None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:41.962343 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:41.978056 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:42.007068 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed883be-dd05-4078-ad2e-72f0bf0d2d77 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:42.022944 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6ac660-138c-48c5-9930-4731699636fd None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:42.045287 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9a4fb0-5f62-4f7b-8344-805978fa0cff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:42.046171 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9a4fb0-5f62-4f7b-8344-805978fa0cff None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:42.046629 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9a4fb0-5f62-4f7b-8344-805978fa0cff None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:42.047077 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9a4fb0-5f62-4f7b-8344-805978fa0cff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:42.060286 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2f0e0e-320b-45a1-ae22-ebd15332d208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:42.060931 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2f0e0e-320b-45a1-ae22-ebd15332d208 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:42.061220 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2f0e0e-320b-45a1-ae22-ebd15332d208 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:42.061500 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2f0e0e-320b-45a1-ae22-ebd15332d208 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:42.072672 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2f0e0e-320b-45a1-ae22-ebd15332d208 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 09:46:42.078099 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-8f2f0e0e-320b-45a1-ae22-ebd15332d208 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 09:46:42.078099 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:42.078099 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:42.078099 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:42.078099 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:42.078099 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:42.078099 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:42.078099 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:42.078099 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:42.078099 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:42.078099 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:42.078099 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:42.078099 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 09:46:42.078099 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 09:46:42.078099 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 09:46:42.078099 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 09:46:42.079278 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 09:46:42.079278 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 09:46:42.079278 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 09:46:42.079278 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 09:46:42.079278 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:42.079278 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:42.079278 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:42.079278 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:42.079278 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:42.079278 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:42.079278 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:42.079278 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:42.079278 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:42.079278 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:42.079278 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:42.079278 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:42.080571 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:42.080571 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:42.080571 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:42.080571 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:42.080571 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 09:46:42.080571 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 09:46:42.080571 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 09:46:42.080571 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 09:46:42.080571 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 09:46:42.080571 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 09:46:42.080571 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 09:46:42.080571 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:46:42.080571 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 319/645] 10.222.0.64 () {56 vars in 1102 bytes} [Tue Aug 2 09:46:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:46:42.080571 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2f0e0e-320b-45a1-ae22-ebd15332d208 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:42.134307 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9a4fb0-5f62-4f7b-8344-805978fa0cff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:42.135782 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9a4fb0-5f62-4f7b-8344-805978fa0cff None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:42.199913 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2f0e0e-320b-45a1-ae22-ebd15332d208 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:42.208834 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:42.222096 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:42.241448 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40d723d-776c-4b14-8b33-b219fdddbbb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:42.242040 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40d723d-776c-4b14-8b33-b219fdddbbb4 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:42.242329 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40d723d-776c-4b14-8b33-b219fdddbbb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:42.242617 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40d723d-776c-4b14-8b33-b219fdddbbb4 None admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:42.243389 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40d723d-776c-4b14-8b33-b219fdddbbb4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:42.243704 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40d723d-776c-4b14-8b33-b219fdddbbb4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:42.245117 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40d723d-776c-4b14-8b33-b219fdddbbb4 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:42.249385 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 327/646] 10.222.0.64 () {58 vars in 1282 bytes} [Tue Aug 2 09:46:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 09:46:42.253412 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9a4fb0-5f62-4f7b-8344-805978fa0cff None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:42.258659 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 320/647] 10.222.0.64 () {56 vars in 1103 bytes} [Tue Aug 2 09:46:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:42.258659 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40d723d-776c-4b14-8b33-b219fdddbbb4 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:42.358468 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9a4fb0-5f62-4f7b-8344-805978fa0cff None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:42.386110 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40d723d-776c-4b14-8b33-b219fdddbbb4 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:42.394952 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90301a3-f16c-4560-8162-244b1708b805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:42.395742 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90301a3-f16c-4560-8162-244b1708b805 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:42.395922 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90301a3-f16c-4560-8162-244b1708b805 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:42.396146 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90301a3-f16c-4560-8162-244b1708b805 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:42.403411 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90301a3-f16c-4560-8162-244b1708b805 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:42.405632 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90301a3-f16c-4560-8162-244b1708b805 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:42.417398 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-f90301a3-f16c-4560-8162-244b1708b805 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 09:46:42.417398 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:42.417398 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:42.417398 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:42.417398 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:42.417398 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:42.417398 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:42.417398 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:42.417398 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:42.417398 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:42.417398 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:42.417398 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:42.417398 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 09:46:42.417398 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 09:46:42.417398 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 09:46:42.418296 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 09:46:42.418296 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:42.418296 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:42.418296 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:42.418296 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:42.418296 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:42.418296 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:42.418296 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:42.418296 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:42.418296 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:42.418296 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:42.418296 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:42.418296 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:42.418296 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:42.418296 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:42.418296 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:42.418296 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:42.419200 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 09:46:42.419200 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 09:46:42.419200 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 09:46:42.419200 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 09:46:42.419200 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 09:46:42.419200 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 09:46:42.419200 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 09:46:42.419200 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:46:42.419200 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 328/648] 10.222.0.64 () {56 vars in 1111 bytes} [Tue Aug 2 09:46:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:46:42.419692 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90301a3-f16c-4560-8162-244b1708b805 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:42.430189 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cde9142-ef5f-439a-9403-5a4cf1caec52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:42.430817 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cde9142-ef5f-439a-9403-5a4cf1caec52 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:42.431105 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cde9142-ef5f-439a-9403-5a4cf1caec52 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:42.431385 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cde9142-ef5f-439a-9403-5a4cf1caec52 None admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:42.432114 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cde9142-ef5f-439a-9403-5a4cf1caec52 None admin] RBAC: Adding query filter params (name=service) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:42.432459 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cde9142-ef5f-439a-9403-5a4cf1caec52 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:42.433829 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cde9142-ef5f-439a-9403-5a4cf1caec52 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:42.446789 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 321/649] 10.222.0.64 () {56 vars in 1103 bytes} [Tue Aug 2 09:46:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:42.446789 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cde9142-ef5f-439a-9403-5a4cf1caec52 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:42.462394 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:42.478158 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:42.539443 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90301a3-f16c-4560-8162-244b1708b805 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:42.555036 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cde9142-ef5f-439a-9403-5a4cf1caec52 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:42.574500 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd2c4ce-d587-41c1-aa95-c2adf28932f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:42.575137 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd2c4ce-d587-41c1-aa95-c2adf28932f0 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:42.575419 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd2c4ce-d587-41c1-aa95-c2adf28932f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:42.575725 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd2c4ce-d587-41c1-aa95-c2adf28932f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:42.586216 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd2c4ce-d587-41c1-aa95-c2adf28932f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:42.587659 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd2c4ce-d587-41c1-aa95-c2adf28932f0 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:42.588321 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f2bfcc-dd9c-48ba-918e-2683d8a4d6f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:42.588957 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f2bfcc-dd9c-48ba-918e-2683d8a4d6f3 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:42.589242 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f2bfcc-dd9c-48ba-918e-2683d8a4d6f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:42.589518 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f2bfcc-dd9c-48ba-918e-2683d8a4d6f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:42.600261 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f2bfcc-dd9c-48ba-918e-2683d8a4d6f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:42.601763 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f2bfcc-dd9c-48ba-918e-2683d8a4d6f3 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:42.602254 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-dbd2c4ce-d587-41c1-aa95-c2adf28932f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:46:42.602254 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:42.602254 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:42.602254 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:42.602254 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:42.602254 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:42.602254 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:42.602254 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:42.602254 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:42.602254 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:42.602254 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:42.602254 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:42.602254 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 09:46:42.602254 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 09:46:42.602254 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 09:46:42.603644 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 09:46:42.603644 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:42.603644 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:42.603644 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:42.603644 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:42.603644 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:42.603644 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:42.603644 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:42.603644 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:42.603644 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:42.603644 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:42.603644 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:42.603644 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:42.603644 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:42.603644 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:42.603644 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:42.603644 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:42.604839 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 09:46:42.604839 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 09:46:42.604839 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:46:42.604839 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:46:42.604839 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 329/650] 10.222.0.64 () {56 vars in 1108 bytes} [Tue Aug 2 09:46:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:46:42.604839 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd2c4ce-d587-41c1-aa95-c2adf28932f0 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:42.620942 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-a0f2bfcc-dd9c-48ba-918e-2683d8a4d6f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:46:42.620942 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:42.620942 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:42.620942 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:42.620942 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:42.620942 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:42.620942 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:42.620942 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:42.620942 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:42.620942 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:42.620942 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:42.620942 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:42.620942 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 09:46:42.620942 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 09:46:42.620942 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 09:46:42.622038 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 09:46:42.622038 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:42.622038 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:42.622038 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:42.622038 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:42.622038 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:42.622038 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:42.622038 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:42.622038 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:42.622038 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:42.622038 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:42.622038 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:42.622038 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:42.622038 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:42.622038 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:42.622038 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:42.622038 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:42.622950 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 09:46:42.622950 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 09:46:42.622950 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:46:42.622950 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:46:42.622950 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 322/651] 10.222.0.64 () {56 vars in 1108 bytes} [Tue Aug 2 09:46:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:46:42.623238 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f2bfcc-dd9c-48ba-918e-2683d8a4d6f3 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:42.696870 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:42.706475 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:42.709936 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd2c4ce-d587-41c1-aa95-c2adf28932f0 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:42.724583 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f2bfcc-dd9c-48ba-918e-2683d8a4d6f3 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:42.743835 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e7c133-ec14-40b2-a988-3040f0845046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:42.744471 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e7c133-ec14-40b2-a988-3040f0845046 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:42.744757 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e7c133-ec14-40b2-a988-3040f0845046 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:42.745029 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e7c133-ec14-40b2-a988-3040f0845046 None admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:42.745775 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e7c133-ec14-40b2-a988-3040f0845046 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:42.746080 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e7c133-ec14-40b2-a988-3040f0845046 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:42.747512 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e7c133-ec14-40b2-a988-3040f0845046 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:42.757980 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba3a978-9024-45cc-b95a-40f6dfe39061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:42.758611 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba3a978-9024-45cc-b95a-40f6dfe39061 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:42.758890 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba3a978-9024-45cc-b95a-40f6dfe39061 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:42.759161 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba3a978-9024-45cc-b95a-40f6dfe39061 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:42.764755 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 330/652] 10.222.0.64 () {56 vars in 1148 bytes} [Tue Aug 2 09:46:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:42.764755 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33554a63-55c0-4f9a-bccc-17b4eec1ad57 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:42.764755 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33554a63-55c0-4f9a-bccc-17b4eec1ad57 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:42.764755 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33554a63-55c0-4f9a-bccc-17b4eec1ad57 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:42.764755 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 331/653] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:46:42 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:46:42.764755 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33554a63-55c0-4f9a-bccc-17b4eec1ad57 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:42.771204 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba3a978-9024-45cc-b95a-40f6dfe39061 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:42.772716 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba3a978-9024-45cc-b95a-40f6dfe39061 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:42.787409 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-dba3a978-9024-45cc-b95a-40f6dfe39061 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:46:42.787409 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:42.787409 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:42.787409 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:42.787409 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:42.787409 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:42.787409 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:42.787409 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:42.787409 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:42.787409 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:42.787409 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:42.787409 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:42.787409 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 09:46:42.787409 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 09:46:42.787409 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 09:46:42.788545 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 09:46:42.788545 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:42.788545 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:42.788545 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:42.788545 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:42.788545 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:42.788545 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:42.788545 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:42.788545 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:42.788545 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:42.788545 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:42.788545 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:42.788545 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:42.788545 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:42.788545 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:42.788545 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:42.788545 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:42.789490 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 09:46:42.789490 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 09:46:42.789490 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:46:42.789490 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:46:42.789490 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 323/654] 10.222.0.64 () {56 vars in 1108 bytes} [Tue Aug 2 09:46:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:46:42.789766 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba3a978-9024-45cc-b95a-40f6dfe39061 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:42.879633 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33554a63-55c0-4f9a-bccc-17b4eec1ad57 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:42.898580 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba3a978-9024-45cc-b95a-40f6dfe39061 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:42.915820 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b98946e-bc32-4e84-84cd-5d38bbf286f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:42.916471 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b98946e-bc32-4e84-84cd-5d38bbf286f7 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:42.916757 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b98946e-bc32-4e84-84cd-5d38bbf286f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:42.917033 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b98946e-bc32-4e84-84cd-5d38bbf286f7 None admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:42.917738 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b98946e-bc32-4e84-84cd-5d38bbf286f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:42.918043 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b98946e-bc32-4e84-84cd-5d38bbf286f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:42.919422 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b98946e-bc32-4e84-84cd-5d38bbf286f7 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:42.930783 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 332/655] 10.222.0.64 () {56 vars in 1109 bytes} [Tue Aug 2 09:46:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:42.931655 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b98946e-bc32-4e84-84cd-5d38bbf286f7 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:42.933228 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0345e8-5c55-499e-ad7d-d3dbde6f7ad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:42.933958 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0345e8-5c55-499e-ad7d-d3dbde6f7ad7 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:42.934315 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0345e8-5c55-499e-ad7d-d3dbde6f7ad7 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:42.934658 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0345e8-5c55-499e-ad7d-d3dbde6f7ad7 None admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:42.935431 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0345e8-5c55-499e-ad7d-d3dbde6f7ad7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:42.935859 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0345e8-5c55-499e-ad7d-d3dbde6f7ad7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:42.937359 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0345e8-5c55-499e-ad7d-d3dbde6f7ad7 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:42.938914 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:42.944467 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:42.950783 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 324/656] 10.222.0.64 () {56 vars in 1109 bytes} [Tue Aug 2 09:46:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:42.952009 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91af9824-b453-4334-b9d4-899f777744b5 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:42.952331 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91af9824-b453-4334-b9d4-899f777744b5 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:42.952610 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91af9824-b453-4334-b9d4-899f777744b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:42.970857 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-91af9824-b453-4334-b9d4-899f777744b5 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:46:42.972076 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-91af9824-b453-4334-b9d4-899f777744b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:46:43.033663 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b98946e-bc32-4e84-84cd-5d38bbf286f7 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:43.038367 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 325/657] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:46:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 09:46:43.039385 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91af9824-b453-4334-b9d4-899f777744b5 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:43.080154 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdddef1-c8d3-4718-9f27-b3d609c3f99a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:43.081145 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdddef1-c8d3-4718-9f27-b3d609c3f99a None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:43.081434 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdddef1-c8d3-4718-9f27-b3d609c3f99a None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:43.081716 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdddef1-c8d3-4718-9f27-b3d609c3f99a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:43.082507 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdddef1-c8d3-4718-9f27-b3d609c3f99a None admin] RBAC: Adding query filter params (role.id=5704af09562a457fa35364e8af9481f1, scope.project.id=390bd7c4f91c4465b0c7ab0f9e436c4d, user.id=10d7de62b9ee4039bdc79627ac7fb499) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:43.082816 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdddef1-c8d3-4718-9f27-b3d609c3f99a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:43.084234 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdddef1-c8d3-4718-9f27-b3d609c3f99a None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:43.090126 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 333/658] 10.222.0.64 () {56 vars in 1374 bytes} [Tue Aug 2 09:46:42 2022] GET /identity/v3/role_assignments?user.id=10d7de62b9ee4039bdc79627ac7fb499&scope.project.id=390bd7c4f91c4465b0c7ab0f9e436c4d&role.id=5704af09562a457fa35364e8af9481f1 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:43.090781 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdddef1-c8d3-4718-9f27-b3d609c3f99a None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:43.145271 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91af9824-b453-4334-b9d4-899f777744b5 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:43.169941 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:43.174256 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:43.188732 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6158b10-5cbc-4e05-a60b-203fcc0e5ee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:43.189533 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6158b10-5cbc-4e05-a60b-203fcc0e5ee2 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:43.189945 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6158b10-5cbc-4e05-a60b-203fcc0e5ee2 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:43.190478 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6158b10-5cbc-4e05-a60b-203fcc0e5ee2 None admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:43.191191 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6158b10-5cbc-4e05-a60b-203fcc0e5ee2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:43.191603 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6158b10-5cbc-4e05-a60b-203fcc0e5ee2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:43.193134 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6158b10-5cbc-4e05-a60b-203fcc0e5ee2 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:43.205295 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdddef1-c8d3-4718-9f27-b3d609c3f99a None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:43.211528 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 326/659] 10.222.0.64 () {56 vars in 1109 bytes} [Tue Aug 2 09:46:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:43.211528 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6158b10-5cbc-4e05-a60b-203fcc0e5ee2 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:43.249696 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f024ef0-6774-4cad-b5d6-d0069b288aca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:43.250348 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f024ef0-6774-4cad-b5d6-d0069b288aca None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:43.251158 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f024ef0-6774-4cad-b5d6-d0069b288aca None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:43.251434 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f024ef0-6774-4cad-b5d6-d0069b288aca None admin] PATH_INFO: `/v3/projects/service` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:43.261167 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f024ef0-6774-4cad-b5d6-d0069b288aca None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:43.263261 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f024ef0-6774-4cad-b5d6-d0069b288aca None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:43.276801 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-0f024ef0-6774-4cad-b5d6-d0069b288aca None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 09:46:43.276801 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:43.276801 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:43.276801 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:43.276801 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:43.276801 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:43.276801 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:43.276801 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:43.276801 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:43.276801 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:43.276801 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:43.276801 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:43.276801 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 09:46:43.276801 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 09:46:43.276801 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 09:46:43.278122 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 09:46:43.278122 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:43.278122 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:43.278122 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:43.278122 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:43.278122 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:43.278122 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:43.278122 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:43.278122 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:43.278122 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:43.278122 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:43.278122 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:43.278122 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:43.278122 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:43.278122 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:43.278122 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:43.278122 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:43.279076 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 09:46:43.279076 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 09:46:43.279076 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 09:46:43.279076 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 09:46:43.279076 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 09:46:43.279076 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 09:46:43.279076 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 09:46:43.279076 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:46:43.282411 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 334/660] 10.222.0.64 () {56 vars in 1111 bytes} [Tue Aug 2 09:46:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:46:43.283295 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f024ef0-6774-4cad-b5d6-d0069b288aca None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:43.338803 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6158b10-5cbc-4e05-a60b-203fcc0e5ee2 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:43.374341 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0969c3-d952-4744-a89d-92cb4166e706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:43.374723 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0969c3-d952-4744-a89d-92cb4166e706 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:43.375004 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0969c3-d952-4744-a89d-92cb4166e706 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:43.375276 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0969c3-d952-4744-a89d-92cb4166e706 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:43.384463 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0969c3-d952-4744-a89d-92cb4166e706 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:43.385953 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0969c3-d952-4744-a89d-92cb4166e706 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:43.394310 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f024ef0-6774-4cad-b5d6-d0069b288aca None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:43.398396 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-ba0969c3-d952-4744-a89d-92cb4166e706 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 02 09:46:43.398396 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:43.398396 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:43.398396 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:43.398396 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:43.398396 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:43.398396 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:43.398396 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:43.398396 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:43.398396 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:43.398396 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:43.398396 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:43.398396 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 09:46:43.398396 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 09:46:43.398396 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 09:46:43.398396 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 09:46:43.399685 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:43.399685 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:43.399685 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 09:46:43.399685 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 09:46:43.399685 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 09:46:43.399685 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 09:46:43.399685 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:43.399685 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:43.399685 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:43.399685 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:43.399685 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:43.399685 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:43.399685 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:43.399685 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:43.399685 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:43.399685 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:43.399685 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:43.399685 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:43.400726 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:43.400726 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:43.400726 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 09:46:43.400726 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 09:46:43.400726 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 09:46:43.400726 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 09:46:43.400726 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 09:46:43.400726 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 09:46:43.400726 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 02 09:46:43.400726 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:46:43.404382 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 327/661] 10.222.0.64 () {56 vars in 1099 bytes} [Tue Aug 2 09:46:43 2022] GET /identity/v3/users/glance => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:46:43.405235 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0969c3-d952-4744-a89d-92cb4166e706 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:43.425916 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:43.432328 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c1deaa-c7a9-48fd-b0cc-53251fedb907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:43.432865 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c1deaa-c7a9-48fd-b0cc-53251fedb907 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:43.433144 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c1deaa-c7a9-48fd-b0cc-53251fedb907 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:43.433415 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c1deaa-c7a9-48fd-b0cc-53251fedb907 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:43.444885 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c1deaa-c7a9-48fd-b0cc-53251fedb907 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 09:46:43.450196 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-f1c1deaa-c7a9-48fd-b0cc-53251fedb907 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 09:46:43.450196 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:43.450196 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:43.450196 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:43.450196 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:43.450196 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:43.450196 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:43.450196 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:43.450196 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:43.450196 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:43.450196 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:43.450196 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:43.450196 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 09:46:43.450196 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 09:46:43.450196 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 09:46:43.450196 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 09:46:43.451164 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 09:46:43.451164 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 09:46:43.451164 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 09:46:43.451164 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 09:46:43.451164 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:43.451164 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:43.451164 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:43.451164 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:43.451164 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:43.451164 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:43.451164 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:43.451164 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:43.451164 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:43.451164 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:43.451164 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:43.451164 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:43.452033 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:43.452033 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:43.452033 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:43.452033 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:43.452033 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 09:46:43.452033 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 09:46:43.452033 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 09:46:43.452033 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 09:46:43.452033 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 09:46:43.452033 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 09:46:43.452033 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 09:46:43.452033 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:46:43.452033 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 335/662] 10.222.0.64 () {56 vars in 1102 bytes} [Tue Aug 2 09:46:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:46:43.452782 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c1deaa-c7a9-48fd-b0cc-53251fedb907 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:43.462673 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:43.522987 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0969c3-d952-4744-a89d-92cb4166e706 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:43.560124 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecf1da6-9c47-4411-813c-ece3bc31dcc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:43.560802 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecf1da6-9c47-4411-813c-ece3bc31dcc1 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:43.561089 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecf1da6-9c47-4411-813c-ece3bc31dcc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:43.561363 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecf1da6-9c47-4411-813c-ece3bc31dcc1 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:43.564096 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c1deaa-c7a9-48fd-b0cc-53251fedb907 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:43.569983 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecf1da6-9c47-4411-813c-ece3bc31dcc1 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:43.571527 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecf1da6-9c47-4411-813c-ece3bc31dcc1 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:43.583290 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-eecf1da6-9c47-4411-813c-ece3bc31dcc1 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 02 09:46:43.583290 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:43.583290 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:43.583290 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:43.583290 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:43.583290 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:43.583290 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:43.583290 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:43.583290 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:43.583290 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:43.583290 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:43.583290 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:43.583290 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 09:46:43.583290 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 09:46:43.583290 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 09:46:43.583290 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 09:46:43.586658 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:43.586658 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:43.586658 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 09:46:43.586658 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 09:46:43.586658 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 09:46:43.586658 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 09:46:43.586658 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:43.586658 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:43.586658 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:43.586658 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:43.586658 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:43.586658 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:43.586658 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:43.586658 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:43.586658 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:43.586658 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:43.586658 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:43.586658 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:43.587898 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:43.587898 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:43.587898 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 09:46:43.587898 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 09:46:43.587898 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 09:46:43.587898 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 09:46:43.587898 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 09:46:43.587898 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 09:46:43.587898 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 02 09:46:43.587898 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:46:43.587898 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 328/663] 10.222.0.64 () {56 vars in 1102 bytes} [Tue Aug 2 09:46:43 2022] GET /identity/v3/users/neutron => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:46:43.587898 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecf1da6-9c47-4411-813c-ece3bc31dcc1 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:43.604736 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c666ad60-3f66-4e5b-9202-9a3c0c8adec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:43.605661 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c666ad60-3f66-4e5b-9202-9a3c0c8adec0 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:43.605661 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c666ad60-3f66-4e5b-9202-9a3c0c8adec0 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:43.605924 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c666ad60-3f66-4e5b-9202-9a3c0c8adec0 None admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:43.606715 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c666ad60-3f66-4e5b-9202-9a3c0c8adec0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:43.606963 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c666ad60-3f66-4e5b-9202-9a3c0c8adec0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:43.608434 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c666ad60-3f66-4e5b-9202-9a3c0c8adec0 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:43.621987 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 336/664] 10.222.0.64 () {56 vars in 1148 bytes} [Tue Aug 2 09:46:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:43.627927 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c666ad60-3f66-4e5b-9202-9a3c0c8adec0 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:43.685430 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:43.699440 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecf1da6-9c47-4411-813c-ece3bc31dcc1 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:43.703337 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:43.735387 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c666ad60-3f66-4e5b-9202-9a3c0c8adec0 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:43.744283 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809c1c0e-a39b-4d85-ad74-fdb95f6e1264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:43.745277 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809c1c0e-a39b-4d85-ad74-fdb95f6e1264 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:43.745575 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809c1c0e-a39b-4d85-ad74-fdb95f6e1264 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:43.745856 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809c1c0e-a39b-4d85-ad74-fdb95f6e1264 None admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:43.746997 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809c1c0e-a39b-4d85-ad74-fdb95f6e1264 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:43.747597 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809c1c0e-a39b-4d85-ad74-fdb95f6e1264 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:43.749228 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809c1c0e-a39b-4d85-ad74-fdb95f6e1264 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:43.770004 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabadb93-0257-4821-b9da-d5c9865ddcaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:43.770624 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabadb93-0257-4821-b9da-d5c9865ddcaf None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:43.770908 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabadb93-0257-4821-b9da-d5c9865ddcaf None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:43.771187 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabadb93-0257-4821-b9da-d5c9865ddcaf None admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:43.771904 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabadb93-0257-4821-b9da-d5c9865ddcaf None admin] RBAC: Adding query filter params (name=service) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:43.772209 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabadb93-0257-4821-b9da-d5c9865ddcaf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:43.773602 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabadb93-0257-4821-b9da-d5c9865ddcaf None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:43.782202 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 329/665] 10.222.0.64 () {56 vars in 1137 bytes} [Tue Aug 2 09:46:43 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:43.782202 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809c1c0e-a39b-4d85-ad74-fdb95f6e1264 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:43.782460 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 337/666] 10.222.0.64 () {56 vars in 1103 bytes} [Tue Aug 2 09:46:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:43.783023 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabadb93-0257-4821-b9da-d5c9865ddcaf None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:43.925359 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabadb93-0257-4821-b9da-d5c9865ddcaf None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:43.926496 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809c1c0e-a39b-4d85-ad74-fdb95f6e1264 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:43.944944 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:43.966365 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2bf1e6-87b1-4460-b24d-3e731bf52a0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:43.966992 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2bf1e6-87b1-4460-b24d-3e731bf52a0f None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:43.967290 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2bf1e6-87b1-4460-b24d-3e731bf52a0f None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:43.967552 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2bf1e6-87b1-4460-b24d-3e731bf52a0f None admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:43.968390 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2bf1e6-87b1-4460-b24d-3e731bf52a0f None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:43.968706 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2bf1e6-87b1-4460-b24d-3e731bf52a0f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:43.969771 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0d3cc9-3657-430c-882a-cf6a0699462a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:43.970394 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0d3cc9-3657-430c-882a-cf6a0699462a None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:43.970682 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0d3cc9-3657-430c-882a-cf6a0699462a None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:43.970792 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2bf1e6-87b1-4460-b24d-3e731bf52a0f None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:43.970956 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0d3cc9-3657-430c-882a-cf6a0699462a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:43.971751 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0d3cc9-3657-430c-882a-cf6a0699462a None admin] RBAC: Adding query filter params (role.id=5704af09562a457fa35364e8af9481f1, scope.project.id=390bd7c4f91c4465b0c7ab0f9e436c4d, user.id=99655e1f9ca24472a86cd26b0dbd1fc2) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:43.972069 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0d3cc9-3657-430c-882a-cf6a0699462a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:43.973534 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0d3cc9-3657-430c-882a-cf6a0699462a None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:43.980019 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 338/667] 10.222.0.64 () {56 vars in 1374 bytes} [Tue Aug 2 09:46:43 2022] GET /identity/v3/role_assignments?user.id=99655e1f9ca24472a86cd26b0dbd1fc2&scope.project.id=390bd7c4f91c4465b0c7ab0f9e436c4d&role.id=5704af09562a457fa35364e8af9481f1 => generated 608 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:43.980851 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0d3cc9-3657-430c-882a-cf6a0699462a None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:44.001467 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 330/668] 10.222.0.64 () {56 vars in 1139 bytes} [Tue Aug 2 09:46:43 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:44.002862 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2bf1e6-87b1-4460-b24d-3e731bf52a0f None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:44.039205 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:44.153297 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2bf1e6-87b1-4460-b24d-3e731bf52a0f None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:44.159859 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0d3cc9-3657-430c-882a-cf6a0699462a None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:44.202640 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c234a4c2-8f6d-4662-958b-52b90ba20c06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:44.203213 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c234a4c2-8f6d-4662-958b-52b90ba20c06 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:44.203526 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c234a4c2-8f6d-4662-958b-52b90ba20c06 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:44.203799 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c234a4c2-8f6d-4662-958b-52b90ba20c06 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:44.207858 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b6ffda-9b2e-446b-a980-f8d3c06bafb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:44.208561 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b6ffda-9b2e-446b-a980-f8d3c06bafb6 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:44.208850 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b6ffda-9b2e-446b-a980-f8d3c06bafb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:44.209134 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b6ffda-9b2e-446b-a980-f8d3c06bafb6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:44.216711 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c234a4c2-8f6d-4662-958b-52b90ba20c06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:44.218304 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c234a4c2-8f6d-4662-958b-52b90ba20c06 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:44.238119 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-c234a4c2-8f6d-4662-958b-52b90ba20c06 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:46:44.238119 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:44.238119 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:44.238119 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:44.238119 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:44.238119 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:44.238119 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:44.238119 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:44.238119 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:44.238119 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:44.238119 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:44.238119 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:44.238119 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 09:46:44.238119 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 09:46:44.238119 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 09:46:44.239078 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 09:46:44.239078 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:44.239078 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:44.239078 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:44.239078 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:44.239078 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:44.239078 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:44.239078 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:44.239078 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:44.239078 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:44.239078 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:44.239078 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:44.239078 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:44.239078 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:44.239078 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:44.239078 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:44.239078 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:44.240009 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 09:46:44.240009 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 09:46:44.240009 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:46:44.240009 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:46:44.240009 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 331/669] 10.222.0.64 () {56 vars in 1108 bytes} [Tue Aug 2 09:46:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:46:44.240330 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c234a4c2-8f6d-4662-958b-52b90ba20c06 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:44.273868 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:44.312179 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b6ffda-9b2e-446b-a980-f8d3c06bafb6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:44.313758 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b6ffda-9b2e-446b-a980-f8d3c06bafb6 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:44.344005 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:44.377369 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c234a4c2-8f6d-4662-958b-52b90ba20c06 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:44.413320 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5832a9-19de-42ed-ad76-8fe4c90b3366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:44.413990 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5832a9-19de-42ed-ad76-8fe4c90b3366 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:44.414271 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5832a9-19de-42ed-ad76-8fe4c90b3366 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:44.414550 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5832a9-19de-42ed-ad76-8fe4c90b3366 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:44.428174 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 339/670] 10.222.0.64 () {58 vars in 1282 bytes} [Tue Aug 2 09:46:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 09:46:44.429060 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b6ffda-9b2e-446b-a980-f8d3c06bafb6 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:44.488646 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5832a9-19de-42ed-ad76-8fe4c90b3366 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:44.490105 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5832a9-19de-42ed-ad76-8fe4c90b3366 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:44.511990 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:44.537169 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b6ffda-9b2e-446b-a980-f8d3c06bafb6 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:44.568603 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:44.578988 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d95fb7e-50d1-4f1e-8fbd-5947bc491cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:44.581318 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d95fb7e-50d1-4f1e-8fbd-5947bc491cba None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:44.581827 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d95fb7e-50d1-4f1e-8fbd-5947bc491cba None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:44.582317 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d95fb7e-50d1-4f1e-8fbd-5947bc491cba None admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:44.583231 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d95fb7e-50d1-4f1e-8fbd-5947bc491cba None admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:44.583748 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d95fb7e-50d1-4f1e-8fbd-5947bc491cba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:44.585422 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d95fb7e-50d1-4f1e-8fbd-5947bc491cba None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:44.598840 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 340/671] 10.222.0.64 () {56 vars in 1109 bytes} [Tue Aug 2 09:46:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:44.600288 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d95fb7e-50d1-4f1e-8fbd-5947bc491cba None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:44.603615 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 332/672] 10.222.0.64 () {58 vars in 1282 bytes} [Tue Aug 2 09:46:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 09:46:44.606853 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5832a9-19de-42ed-ad76-8fe4c90b3366 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:44.725069 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d95fb7e-50d1-4f1e-8fbd-5947bc491cba None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:44.736504 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5832a9-19de-42ed-ad76-8fe4c90b3366 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:44.764542 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687b4efc-1a16-48ba-bb30-b15ef53d6662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:44.765600 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687b4efc-1a16-48ba-bb30-b15ef53d6662 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:44.766766 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687b4efc-1a16-48ba-bb30-b15ef53d6662 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:44.767271 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687b4efc-1a16-48ba-bb30-b15ef53d6662 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:44.776832 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:44.782044 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fbcd73-c6e9-4a2d-b524-4383b3a6d6cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:44.783149 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fbcd73-c6e9-4a2d-b524-4383b3a6d6cf None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:44.783506 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fbcd73-c6e9-4a2d-b524-4383b3a6d6cf None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:44.783753 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fbcd73-c6e9-4a2d-b524-4383b3a6d6cf None admin] PATH_INFO: `/v3/users/swift` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:44.785021 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687b4efc-1a16-48ba-bb30-b15ef53d6662 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:44.786490 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687b4efc-1a16-48ba-bb30-b15ef53d6662 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:44.794858 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fbcd73-c6e9-4a2d-b524-4383b3a6d6cf None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:44.796631 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fbcd73-c6e9-4a2d-b524-4383b3a6d6cf None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:44.807072 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-687b4efc-1a16-48ba-bb30-b15ef53d6662 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:46:44.807072 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:44.807072 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:44.807072 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:44.807072 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:44.807072 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:44.807072 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:44.807072 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:44.807072 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:44.807072 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:44.807072 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:44.807072 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:44.807072 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 09:46:44.807072 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 09:46:44.807072 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 09:46:44.808186 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 09:46:44.808186 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:44.808186 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:44.808186 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:44.808186 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:44.808186 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:44.808186 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:44.808186 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:44.808186 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:44.808186 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:44.808186 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:44.808186 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:44.808186 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:44.808186 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:44.808186 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:44.808186 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:44.808186 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:44.809148 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 09:46:44.809148 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 09:46:44.809148 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:46:44.809148 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:46:44.810885 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-83fbcd73-c6e9-4a2d-b524-4383b3a6d6cf None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 02 09:46:44.810885 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:44.810885 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:44.810885 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:44.810885 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:44.810885 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:44.810885 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:44.810885 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:44.810885 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:44.810885 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:44.810885 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:44.810885 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:44.810885 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 09:46:44.810885 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 09:46:44.810885 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 09:46:44.810885 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 09:46:44.811811 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:44.811811 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:44.811811 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 09:46:44.811811 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 09:46:44.811811 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 09:46:44.811811 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 09:46:44.811811 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:44.811811 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:44.811811 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:44.811811 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:44.811811 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:44.811811 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:44.811811 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:44.811811 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:44.811811 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:44.811811 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:44.811811 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:44.811811 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:44.813883 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:44.813883 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:44.813883 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 09:46:44.813883 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 09:46:44.813883 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 09:46:44.813883 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 09:46:44.813883 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 09:46:44.813883 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 09:46:44.813883 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 02 09:46:44.813883 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:46:44.813883 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 333/673] 10.222.0.64 () {56 vars in 1096 bytes} [Tue Aug 2 09:46:44 2022] GET /identity/v3/users/swift => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:46:44.813883 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fbcd73-c6e9-4a2d-b524-4383b3a6d6cf None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:44.815252 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 341/674] 10.222.0.64 () {56 vars in 1108 bytes} [Tue Aug 2 09:46:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:46:44.820352 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687b4efc-1a16-48ba-bb30-b15ef53d6662 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:44.856206 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:44.939832 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fbcd73-c6e9-4a2d-b524-4383b3a6d6cf None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:44.963865 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687b4efc-1a16-48ba-bb30-b15ef53d6662 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:44.977742 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398fc67c-b80b-462c-b8aa-d26c66c25e12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:44.978509 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398fc67c-b80b-462c-b8aa-d26c66c25e12 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:44.978901 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398fc67c-b80b-462c-b8aa-d26c66c25e12 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:44.979275 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398fc67c-b80b-462c-b8aa-d26c66c25e12 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:44.991063 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398fc67c-b80b-462c-b8aa-d26c66c25e12 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:44.992757 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398fc67c-b80b-462c-b8aa-d26c66c25e12 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:45.008015 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-398fc67c-b80b-462c-b8aa-d26c66c25e12 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:46:45.008015 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:45.008015 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:45.008015 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:45.008015 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:45.008015 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:45.008015 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:45.008015 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:45.008015 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:45.008015 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:45.008015 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:45.008015 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:45.008015 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 09:46:45.008015 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 09:46:45.008015 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 09:46:45.009199 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 09:46:45.009199 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:45.009199 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:45.009199 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:45.009199 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:45.009199 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:45.009199 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:45.009199 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:45.009199 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:45.009199 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:45.009199 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:45.009199 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:45.009199 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:45.009199 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:45.009199 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:45.009199 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:45.009199 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:45.010339 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 09:46:45.010339 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 09:46:45.010339 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:46:45.010339 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:46:45.011779 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 334/675] 10.222.0.64 () {56 vars in 1108 bytes} [Tue Aug 2 09:46:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:46:45.012275 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958e8ed5-2ffa-4855-8db0-5f2e97ef385d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:45.012620 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398fc67c-b80b-462c-b8aa-d26c66c25e12 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:45.013263 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958e8ed5-2ffa-4855-8db0-5f2e97ef385d None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:45.013701 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958e8ed5-2ffa-4855-8db0-5f2e97ef385d None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:45.014117 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958e8ed5-2ffa-4855-8db0-5f2e97ef385d None admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:45.015121 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958e8ed5-2ffa-4855-8db0-5f2e97ef385d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:45.015869 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958e8ed5-2ffa-4855-8db0-5f2e97ef385d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:45.018980 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958e8ed5-2ffa-4855-8db0-5f2e97ef385d None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:45.044963 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 342/676] 10.222.0.64 () {56 vars in 1135 bytes} [Tue Aug 2 09:46:44 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:45.046017 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958e8ed5-2ffa-4855-8db0-5f2e97ef385d None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:45.114574 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:45.117696 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:45.136970 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398fc67c-b80b-462c-b8aa-d26c66c25e12 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:45.175699 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95710edb-b1b2-481c-9045-d42abd1b147d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:45.176639 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95710edb-b1b2-481c-9045-d42abd1b147d None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:45.176948 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95710edb-b1b2-481c-9045-d42abd1b147d None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:45.177350 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95710edb-b1b2-481c-9045-d42abd1b147d None admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:45.178162 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95710edb-b1b2-481c-9045-d42abd1b147d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:45.178562 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95710edb-b1b2-481c-9045-d42abd1b147d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:45.180031 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95710edb-b1b2-481c-9045-d42abd1b147d None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:45.187111 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958e8ed5-2ffa-4855-8db0-5f2e97ef385d None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:45.198249 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 335/677] 10.222.0.64 () {56 vars in 1109 bytes} [Tue Aug 2 09:46:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:45.199416 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95710edb-b1b2-481c-9045-d42abd1b147d None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:45.234430 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cc8cfe-2c3a-440c-8eab-ca5f9dfc043e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:45.236496 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cc8cfe-2c3a-440c-8eab-ca5f9dfc043e None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:45.237080 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cc8cfe-2c3a-440c-8eab-ca5f9dfc043e None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:45.238446 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cc8cfe-2c3a-440c-8eab-ca5f9dfc043e None admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:45.239475 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cc8cfe-2c3a-440c-8eab-ca5f9dfc043e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:45.240694 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cc8cfe-2c3a-440c-8eab-ca5f9dfc043e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:45.242769 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cc8cfe-2c3a-440c-8eab-ca5f9dfc043e None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:45.255991 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 343/678] 10.222.0.64 () {56 vars in 1109 bytes} [Tue Aug 2 09:46:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:45.257209 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cc8cfe-2c3a-440c-8eab-ca5f9dfc043e None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:45.326917 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95710edb-b1b2-481c-9045-d42abd1b147d None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:45.373349 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0946735c-b6d2-4906-a31c-4fd90d6aa987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:45.373972 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0946735c-b6d2-4906-a31c-4fd90d6aa987 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:45.374275 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0946735c-b6d2-4906-a31c-4fd90d6aa987 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:45.374379 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cc8cfe-2c3a-440c-8eab-ca5f9dfc043e None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:45.374541 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0946735c-b6d2-4906-a31c-4fd90d6aa987 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:45.384129 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:45.390832 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:45.411672 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96346e9c-0cd8-4537-ab18-04dfc63abfb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:45.412393 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96346e9c-0cd8-4537-ab18-04dfc63abfb5 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:45.412684 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96346e9c-0cd8-4537-ab18-04dfc63abfb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:45.412962 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96346e9c-0cd8-4537-ab18-04dfc63abfb5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:45.419849 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96346e9c-0cd8-4537-ab18-04dfc63abfb5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:45.421341 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96346e9c-0cd8-4537-ab18-04dfc63abfb5 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:45.434236 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-96346e9c-0cd8-4537-ab18-04dfc63abfb5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 09:46:45.434236 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:45.434236 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:45.434236 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:45.434236 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:45.434236 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:45.434236 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:45.434236 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:45.434236 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:45.434236 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:45.434236 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:45.434236 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:45.434236 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 09:46:45.434236 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 09:46:45.434236 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 09:46:45.435145 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 09:46:45.435145 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:45.435145 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:45.435145 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:45.435145 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:45.435145 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:45.435145 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:45.435145 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:45.435145 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:45.435145 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:45.435145 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:45.435145 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:45.435145 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:45.435145 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:45.435145 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:45.435145 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:45.435145 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:45.436090 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 09:46:45.436090 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 09:46:45.436090 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 09:46:45.436090 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 09:46:45.436090 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 09:46:45.436090 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 09:46:45.436090 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 09:46:45.436090 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:46:45.436090 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 344/679] 10.222.0.64 () {56 vars in 1111 bytes} [Tue Aug 2 09:46:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:46:45.437164 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96346e9c-0cd8-4537-ab18-04dfc63abfb5 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:45.451735 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0946735c-b6d2-4906-a31c-4fd90d6aa987 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:45.453252 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0946735c-b6d2-4906-a31c-4fd90d6aa987 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:45.543756 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96346e9c-0cd8-4537-ab18-04dfc63abfb5 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:45.561030 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 336/680] 10.222.0.64 () {58 vars in 1282 bytes} [Tue Aug 2 09:46:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 09:46:45.565981 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb20bf3-2c16-4e3b-bcc9-f025e1f74aac None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:45.565981 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb20bf3-2c16-4e3b-bcc9-f025e1f74aac None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:45.565981 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb20bf3-2c16-4e3b-bcc9-f025e1f74aac None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:45.565981 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 337/681] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:46:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:46:45.565981 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb20bf3-2c16-4e3b-bcc9-f025e1f74aac None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:45.582353 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4b9ab2-b774-4c3c-94fa-2c23353517bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:45.582923 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4b9ab2-b774-4c3c-94fa-2c23353517bc None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:45.583208 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4b9ab2-b774-4c3c-94fa-2c23353517bc None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:45.583487 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4b9ab2-b774-4c3c-94fa-2c23353517bc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:45.590484 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4b9ab2-b774-4c3c-94fa-2c23353517bc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:45.591933 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4b9ab2-b774-4c3c-94fa-2c23353517bc None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:45.604042 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-db4b9ab2-b774-4c3c-94fa-2c23353517bc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 09:46:45.604042 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:45.604042 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:45.604042 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:45.604042 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:45.604042 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:45.604042 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:45.604042 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:45.604042 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:45.604042 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:45.604042 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:45.604042 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:45.604042 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 09:46:45.604042 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 09:46:45.604042 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 09:46:45.605271 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 09:46:45.605271 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:45.605271 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:45.605271 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:45.605271 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:45.605271 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:45.605271 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:45.605271 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:45.605271 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:45.605271 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:45.605271 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:45.605271 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:45.605271 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:45.605271 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:45.605271 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:45.605271 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:45.605271 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:45.606200 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 09:46:45.606200 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 09:46:45.606200 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 09:46:45.606200 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 09:46:45.606200 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 09:46:45.606200 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 09:46:45.606200 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 09:46:45.606200 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:46:45.606200 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 345/682] 10.222.0.64 () {56 vars in 1111 bytes} [Tue Aug 2 09:46:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:46:45.606200 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4b9ab2-b774-4c3c-94fa-2c23353517bc None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:45.619376 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:45.637094 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:45.680141 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb20bf3-2c16-4e3b-bcc9-f025e1f74aac None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:45.713423 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4b9ab2-b774-4c3c-94fa-2c23353517bc None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:45.714925 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ed664d-ac7d-4223-8053-0f98a85fad70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:45.715523 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ed664d-ac7d-4223-8053-0f98a85fad70 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:45.715819 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ed664d-ac7d-4223-8053-0f98a85fad70 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:45.716089 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ed664d-ac7d-4223-8053-0f98a85fad70 None admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:45.716877 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ed664d-ac7d-4223-8053-0f98a85fad70 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:45.717190 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ed664d-ac7d-4223-8053-0f98a85fad70 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:45.718644 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ed664d-ac7d-4223-8053-0f98a85fad70 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:45.730559 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 338/683] 10.222.0.64 () {56 vars in 1148 bytes} [Tue Aug 2 09:46:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:45.731861 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac6f799-f087-49c1-94a1-e7f77dc5888b None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:45.732140 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac6f799-f087-49c1-94a1-e7f77dc5888b None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:45.732450 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac6f799-f087-49c1-94a1-e7f77dc5888b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:45.751799 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-cac6f799-f087-49c1-94a1-e7f77dc5888b None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:46:45.753004 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-cac6f799-f087-49c1-94a1-e7f77dc5888b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:46:45.758034 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f803de-8b7a-4790-a795-2c0b58d73187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:45.759258 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f803de-8b7a-4790-a795-2c0b58d73187 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:45.759731 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f803de-8b7a-4790-a795-2c0b58d73187 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:45.760277 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f803de-8b7a-4790-a795-2c0b58d73187 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:45.772214 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f803de-8b7a-4790-a795-2c0b58d73187 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:45.774037 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f803de-8b7a-4790-a795-2c0b58d73187 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:45.791932 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-12f803de-8b7a-4790-a795-2c0b58d73187 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:46:45.791932 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:45.791932 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:45.791932 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:45.791932 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:45.791932 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:45.791932 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:45.791932 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:45.791932 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:45.791932 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:45.791932 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:45.791932 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:45.791932 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 09:46:45.791932 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 09:46:45.791932 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 09:46:45.793041 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 09:46:45.793041 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:45.793041 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:45.793041 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:45.793041 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:45.793041 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:45.793041 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:45.793041 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:45.793041 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:45.793041 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:45.793041 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:45.793041 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:45.793041 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:45.793041 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:45.793041 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:45.793041 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:45.793041 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:45.794177 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 09:46:45.794177 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 09:46:45.794177 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:46:45.794177 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:46:45.795791 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 346/684] 10.222.0.64 () {56 vars in 1108 bytes} [Tue Aug 2 09:46:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:46:45.796766 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f803de-8b7a-4790-a795-2c0b58d73187 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:45.828540 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 339/685] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:46:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 09:46:45.829808 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac6f799-f087-49c1-94a1-e7f77dc5888b None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:45.869416 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:45.903406 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:45.924375 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f803de-8b7a-4790-a795-2c0b58d73187 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:45.944660 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac6f799-f087-49c1-94a1-e7f77dc5888b None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:45.960440 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea2558a-1cd2-4756-b820-71a9d8104037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:45.961083 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea2558a-1cd2-4756-b820-71a9d8104037 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:45.961366 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea2558a-1cd2-4756-b820-71a9d8104037 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:45.961644 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea2558a-1cd2-4756-b820-71a9d8104037 None admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:45.962391 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea2558a-1cd2-4756-b820-71a9d8104037 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:45.962698 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea2558a-1cd2-4756-b820-71a9d8104037 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:45.964124 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea2558a-1cd2-4756-b820-71a9d8104037 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:45.977364 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 347/686] 10.222.0.64 () {56 vars in 1148 bytes} [Tue Aug 2 09:46:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:45.978085 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea2558a-1cd2-4756-b820-71a9d8104037 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:45.985583 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81a5c79-6b1f-4c3f-8f0c-c3082e2e5a7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:45.986203 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81a5c79-6b1f-4c3f-8f0c-c3082e2e5a7a None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:45.986495 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81a5c79-6b1f-4c3f-8f0c-c3082e2e5a7a None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:45.986775 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81a5c79-6b1f-4c3f-8f0c-c3082e2e5a7a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:45.987584 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81a5c79-6b1f-4c3f-8f0c-c3082e2e5a7a None admin] RBAC: Adding query filter params (role.id=5704af09562a457fa35364e8af9481f1, scope.project.id=390bd7c4f91c4465b0c7ab0f9e436c4d, user.id=04fb1dbe260a4721aaa0bd0493260d33) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:45.987900 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81a5c79-6b1f-4c3f-8f0c-c3082e2e5a7a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:45.989369 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81a5c79-6b1f-4c3f-8f0c-c3082e2e5a7a None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:45.995697 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 340/687] 10.222.0.64 () {56 vars in 1374 bytes} [Tue Aug 2 09:46:45 2022] GET /identity/v3/role_assignments?user.id=04fb1dbe260a4721aaa0bd0493260d33&scope.project.id=390bd7c4f91c4465b0c7ab0f9e436c4d&role.id=5704af09562a457fa35364e8af9481f1 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:45.996516 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81a5c79-6b1f-4c3f-8f0c-c3082e2e5a7a None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:46.116125 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea2558a-1cd2-4756-b820-71a9d8104037 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:46.117022 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:46.121422 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81a5c79-6b1f-4c3f-8f0c-c3082e2e5a7a None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:46.160111 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2c11b0-1b41-438e-911b-cd6ea54e37a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:46.161040 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2c11b0-1b41-438e-911b-cd6ea54e37a7 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:46.161532 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2c11b0-1b41-438e-911b-cd6ea54e37a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:46.162004 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2c11b0-1b41-438e-911b-cd6ea54e37a7 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:46.162835 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2c11b0-1b41-438e-911b-cd6ea54e37a7 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:46.164449 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2c11b0-1b41-438e-911b-cd6ea54e37a7 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:46.171440 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b76aa3d-f8d0-4803-b290-cf637dcea201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:46.172282 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b76aa3d-f8d0-4803-b290-cf637dcea201 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:46.172723 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b76aa3d-f8d0-4803-b290-cf637dcea201 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:46.173150 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b76aa3d-f8d0-4803-b290-cf637dcea201 None admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:46.173938 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b76aa3d-f8d0-4803-b290-cf637dcea201 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:46.174318 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b76aa3d-f8d0-4803-b290-cf637dcea201 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:46.175899 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b76aa3d-f8d0-4803-b290-cf637dcea201 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:46.178737 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-1a2c11b0-1b41-438e-911b-cd6ea54e37a7 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 02 09:46:46.178737 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:46.178737 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 09:46:46.178737 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 09:46:46.178737 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 09:46:46.178737 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 09:46:46.178737 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 09:46:46.178737 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 09:46:46.178737 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 02 09:46:46.178737 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:46:46.178737 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 09:46:46.178737 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:46:46.178737 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:46.178737 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:46.178737 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:46.178737 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:46.179991 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:46.179991 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:46.179991 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:46.179991 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:46.179991 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:46.179991 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:46.179991 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:46.179991 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 09:46:46.179991 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 09:46:46.179991 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 09:46:46.179991 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 09:46:46.179991 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:46.179991 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:46.179991 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:46.179991 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:46.179991 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:46.181109 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:46.181109 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:46.181109 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:46.181109 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:46.181109 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:46.181109 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:46.181109 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:46.181109 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:46.181109 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:46.181109 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:46.181109 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:46.181109 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 09:46:46.181109 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 09:46:46.181109 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 02 09:46:46.181109 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:46:46.181109 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 341/688] 10.222.0.64 () {56 vars in 1129 bytes} [Tue Aug 2 09:46:45 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:46:46.182189 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2c11b0-1b41-438e-911b-cd6ea54e37a7 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:46.190899 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:46.195892 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 348/689] 10.222.0.64 () {56 vars in 1109 bytes} [Tue Aug 2 09:46:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:46.197640 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173b46f9-f99d-48b2-92fd-b845caac8512 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:46.198297 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173b46f9-f99d-48b2-92fd-b845caac8512 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:46.198908 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173b46f9-f99d-48b2-92fd-b845caac8512 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:46.201108 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 349/690] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:46:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:46:46.203016 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173b46f9-f99d-48b2-92fd-b845caac8512 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:46.297565 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2c11b0-1b41-438e-911b-cd6ea54e37a7 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:46.333111 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173b46f9-f99d-48b2-92fd-b845caac8512 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:46.337927 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c4cc2b-4280-4d41-b7af-d092979de18c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:46.338556 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c4cc2b-4280-4d41-b7af-d092979de18c None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:46.338842 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c4cc2b-4280-4d41-b7af-d092979de18c None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:46.339122 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c4cc2b-4280-4d41-b7af-d092979de18c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:46.339921 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c4cc2b-4280-4d41-b7af-d092979de18c None admin] RBAC: Adding query filter params (role.id=5704af09562a457fa35364e8af9481f1, scope.project.id=390bd7c4f91c4465b0c7ab0f9e436c4d, user.id=ec31b188970947efb2ed1d4d9a31cadf) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:46.340372 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c4cc2b-4280-4d41-b7af-d092979de18c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:46.341860 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c4cc2b-4280-4d41-b7af-d092979de18c None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:46.348052 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 342/691] 10.222.0.64 () {56 vars in 1374 bytes} [Tue Aug 2 09:46:46 2022] GET /identity/v3/role_assignments?user.id=ec31b188970947efb2ed1d4d9a31cadf&scope.project.id=390bd7c4f91c4465b0c7ab0f9e436c4d&role.id=5704af09562a457fa35364e8af9481f1 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:46.348695 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c4cc2b-4280-4d41-b7af-d092979de18c None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:46.385128 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:46.392720 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf21d92-b13e-4204-b01d-194dfbd40040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:46.393908 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf21d92-b13e-4204-b01d-194dfbd40040 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:46.395280 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf21d92-b13e-4204-b01d-194dfbd40040 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:46.395814 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf21d92-b13e-4204-b01d-194dfbd40040 None admin] PATH_INFO: `/v3/services` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:46.396764 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf21d92-b13e-4204-b01d-194dfbd40040 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:46.397287 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf21d92-b13e-4204-b01d-194dfbd40040 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:46.398933 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf21d92-b13e-4204-b01d-194dfbd40040 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:46.411029 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 350/692] 10.222.0.64 () {56 vars in 1124 bytes} [Tue Aug 2 09:46:46 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:46.411029 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4922d1cf-72b1-412b-9398-e3a45a7ae2e7 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:46.411029 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4922d1cf-72b1-412b-9398-e3a45a7ae2e7 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:46.411029 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4922d1cf-72b1-412b-9398-e3a45a7ae2e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:46.435610 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-4922d1cf-72b1-412b-9398-e3a45a7ae2e7 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:46:46.437638 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-4922d1cf-72b1-412b-9398-e3a45a7ae2e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:46:46.496635 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c4cc2b-4280-4d41-b7af-d092979de18c None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:46.514714 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:46.529887 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 351/693] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:46:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 09:46:46.536398 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4922d1cf-72b1-412b-9398-e3a45a7ae2e7 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:46.564186 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f630d1f-f599-4cf8-99e0-1d96aa280412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:46.564899 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f630d1f-f599-4cf8-99e0-1d96aa280412 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:46.565182 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f630d1f-f599-4cf8-99e0-1d96aa280412 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:46.565452 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f630d1f-f599-4cf8-99e0-1d96aa280412 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:46.573381 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f630d1f-f599-4cf8-99e0-1d96aa280412 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:46.575126 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f630d1f-f599-4cf8-99e0-1d96aa280412 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:46.587922 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-7f630d1f-f599-4cf8-99e0-1d96aa280412 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 09:46:46.587922 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:46.587922 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:46.587922 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:46.587922 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:46.587922 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:46.587922 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:46.587922 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:46.587922 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:46.587922 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:46.587922 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:46.587922 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:46.587922 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 09:46:46.587922 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 09:46:46.587922 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 09:46:46.589197 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 09:46:46.589197 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:46.589197 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:46.589197 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:46.589197 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:46.589197 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:46.589197 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:46.589197 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:46.589197 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:46.589197 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:46.589197 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:46.589197 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:46.589197 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:46.589197 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:46.589197 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:46.589197 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:46.589197 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:46.590304 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 09:46:46.590304 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 09:46:46.590304 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 09:46:46.590304 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 09:46:46.590304 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 09:46:46.590304 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 09:46:46.590304 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 09:46:46.590304 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:46:46.592085 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 343/694] 10.222.0.64 () {56 vars in 1111 bytes} [Tue Aug 2 09:46:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:46:46.593055 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f630d1f-f599-4cf8-99e0-1d96aa280412 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:46.678683 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4922d1cf-72b1-412b-9398-e3a45a7ae2e7 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:46.691627 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:46.712308 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f630d1f-f599-4cf8-99e0-1d96aa280412 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:46.730626 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaeed20-3b1f-4634-8b02-ec842100356c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:46.731621 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaeed20-3b1f-4634-8b02-ec842100356c None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:46.731911 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaeed20-3b1f-4634-8b02-ec842100356c None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:46.732193 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaeed20-3b1f-4634-8b02-ec842100356c None admin] PATH_INFO: `/v3/services` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:46.733094 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afaeed20-3b1f-4634-8b02-ec842100356c None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:46.733405 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afaeed20-3b1f-4634-8b02-ec842100356c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:46.735319 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afaeed20-3b1f-4634-8b02-ec842100356c None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:46.748485 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 352/695] 10.222.0.64 () {56 vars in 1124 bytes} [Tue Aug 2 09:46:46 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:46.748485 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaeed20-3b1f-4634-8b02-ec842100356c None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:46.773073 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1eed22-bc88-4416-a426-25c432a5ab47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:46.774064 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1eed22-bc88-4416-a426-25c432a5ab47 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:46.774569 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1eed22-bc88-4416-a426-25c432a5ab47 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:46.775024 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1eed22-bc88-4416-a426-25c432a5ab47 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:46.800512 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1eed22-bc88-4416-a426-25c432a5ab47 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 09:46:46.806422 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-9b1eed22-bc88-4416-a426-25c432a5ab47 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 09:46:46.806422 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:46.806422 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:46.806422 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:46.806422 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:46.806422 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:46.806422 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:46.806422 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:46.806422 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:46.806422 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:46.806422 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:46.806422 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:46.806422 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 09:46:46.806422 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 09:46:46.806422 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 09:46:46.806422 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 09:46:46.807382 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 09:46:46.807382 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 09:46:46.807382 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 09:46:46.807382 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 09:46:46.807382 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:46.807382 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:46.807382 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:46.807382 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:46.807382 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:46.807382 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:46.807382 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:46.807382 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:46.807382 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:46.807382 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:46.807382 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:46.807382 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:46.808241 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:46.808241 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:46.808241 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:46.808241 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:46.808241 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 09:46:46.808241 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 09:46:46.808241 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 09:46:46.808241 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 09:46:46.808241 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 09:46:46.808241 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 09:46:46.808241 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 09:46:46.808241 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:46:46.817340 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 344/696] 10.222.0.64 () {56 vars in 1096 bytes} [Tue Aug 2 09:46:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:46:46.818600 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:46.823818 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1eed22-bc88-4416-a426-25c432a5ab47 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:46.877667 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaeed20-3b1f-4634-8b02-ec842100356c None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:46.928338 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c13da5c-781f-40c3-a5a5-00e8b6ba3ab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:46.928338 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c13da5c-781f-40c3-a5a5-00e8b6ba3ab2 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:46.928338 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c13da5c-781f-40c3-a5a5-00e8b6ba3ab2 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:46.928338 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c13da5c-781f-40c3-a5a5-00e8b6ba3ab2 None admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:46.928834 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c13da5c-781f-40c3-a5a5-00e8b6ba3ab2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:46.928834 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c13da5c-781f-40c3-a5a5-00e8b6ba3ab2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:46.928834 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c13da5c-781f-40c3-a5a5-00e8b6ba3ab2 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:46.939240 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 353/697] 10.222.0.64 () {56 vars in 1148 bytes} [Tue Aug 2 09:46:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:46.946634 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c13da5c-781f-40c3-a5a5-00e8b6ba3ab2 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:47.013063 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1eed22-bc88-4416-a426-25c432a5ab47 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:47.061451 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d526a6-67d3-4efe-8ca3-59e8c401c56c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:47.063649 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d526a6-67d3-4efe-8ca3-59e8c401c56c None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:47.064149 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d526a6-67d3-4efe-8ca3-59e8c401c56c None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:47.064891 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d526a6-67d3-4efe-8ca3-59e8c401c56c None admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:47.066521 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d526a6-67d3-4efe-8ca3-59e8c401c56c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:47.067042 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d526a6-67d3-4efe-8ca3-59e8c401c56c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:47.068183 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c13da5c-781f-40c3-a5a5-00e8b6ba3ab2 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:47.068903 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d526a6-67d3-4efe-8ca3-59e8c401c56c None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:47.081613 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 345/698] 10.222.0.64 () {56 vars in 1099 bytes} [Tue Aug 2 09:46:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:46:47.087498 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d526a6-67d3-4efe-8ca3-59e8c401c56c None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:47.088992 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:47.106962 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:47.109456 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699ad6d0-72ea-4737-9ec3-f29e950e42fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:47.111222 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699ad6d0-72ea-4737-9ec3-f29e950e42fb None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:47.111645 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699ad6d0-72ea-4737-9ec3-f29e950e42fb None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:47.112052 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699ad6d0-72ea-4737-9ec3-f29e950e42fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:47.113049 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699ad6d0-72ea-4737-9ec3-f29e950e42fb None admin] RBAC: Adding query filter params (role.id=5704af09562a457fa35364e8af9481f1, scope.project.id=390bd7c4f91c4465b0c7ab0f9e436c4d, user.id=6a371e3df3764a4cbb1b8368aa74bd65) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:47.113538 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699ad6d0-72ea-4737-9ec3-f29e950e42fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:47.115238 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699ad6d0-72ea-4737-9ec3-f29e950e42fb None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:47.122117 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 354/699] 10.222.0.64 () {56 vars in 1374 bytes} [Tue Aug 2 09:46:46 2022] GET /identity/v3/role_assignments?user.id=6a371e3df3764a4cbb1b8368aa74bd65&scope.project.id=390bd7c4f91c4465b0c7ab0f9e436c4d&role.id=5704af09562a457fa35364e8af9481f1 => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:47.266840 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d526a6-67d3-4efe-8ca3-59e8c401c56c None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:47.322109 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86b160b-0778-417f-ad7b-79178dfe4d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:47.323067 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86b160b-0778-417f-ad7b-79178dfe4d44 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:47.323502 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86b160b-0778-417f-ad7b-79178dfe4d44 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:47.323932 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86b160b-0778-417f-ad7b-79178dfe4d44 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:47.341003 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86b160b-0778-417f-ad7b-79178dfe4d44 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:47.342750 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86b160b-0778-417f-ad7b-79178dfe4d44 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:47.359137 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-f86b160b-0778-417f-ad7b-79178dfe4d44 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:46:47.359137 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:47.359137 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:47.359137 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:47.359137 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:47.359137 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:47.359137 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:47.359137 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:47.359137 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:47.359137 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:47.359137 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:47.359137 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:47.359137 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 09:46:47.359137 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 09:46:47.359137 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 09:46:47.360953 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 09:46:47.360953 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:47.360953 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:47.360953 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:47.360953 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:47.360953 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:47.360953 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:47.360953 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:47.360953 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:47.360953 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:47.360953 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:47.360953 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:47.360953 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:47.360953 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:47.360953 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:47.360953 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:47.360953 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:47.362154 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 09:46:47.362154 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 09:46:47.362154 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:46:47.362154 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:46:47.364014 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 346/700] 10.222.0.64 () {56 vars in 1108 bytes} [Tue Aug 2 09:46:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:46:47.369623 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699ad6d0-72ea-4737-9ec3-f29e950e42fb None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:47.483085 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699ad6d0-72ea-4737-9ec3-f29e950e42fb None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:47.519414 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fcebb4-406f-4a6c-bc96-962639d823d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:47.520314 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fcebb4-406f-4a6c-bc96-962639d823d6 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:47.520747 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fcebb4-406f-4a6c-bc96-962639d823d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:47.521194 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fcebb4-406f-4a6c-bc96-962639d823d6 None admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:47.522065 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fcebb4-406f-4a6c-bc96-962639d823d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:47.522534 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fcebb4-406f-4a6c-bc96-962639d823d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:47.524118 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fcebb4-406f-4a6c-bc96-962639d823d6 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:47.657129 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 355/701] 10.222.0.64 () {56 vars in 1109 bytes} [Tue Aug 2 09:46:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:47.663058 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86b160b-0778-417f-ad7b-79178dfe4d44 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:47.727681 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:47.774941 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86b160b-0778-417f-ad7b-79178dfe4d44 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:47.813318 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3513bdd4-7a14-468d-bcc6-a74b5ac6005b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:47.814262 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3513bdd4-7a14-468d-bcc6-a74b5ac6005b None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:47.814694 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3513bdd4-7a14-468d-bcc6-a74b5ac6005b None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:47.815128 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3513bdd4-7a14-468d-bcc6-a74b5ac6005b None admin] PATH_INFO: `/v3/users/nova` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:47.823468 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3513bdd4-7a14-468d-bcc6-a74b5ac6005b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:47.825159 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3513bdd4-7a14-468d-bcc6-a74b5ac6005b None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:47.838290 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-3513bdd4-7a14-468d-bcc6-a74b5ac6005b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 02 09:46:47.838290 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:47.838290 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:47.838290 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:47.838290 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:47.838290 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:47.838290 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:47.838290 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:47.838290 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:47.838290 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:47.838290 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:47.838290 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:47.838290 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 09:46:47.838290 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 09:46:47.838290 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 09:46:47.838290 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 09:46:47.839478 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:47.839478 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:47.839478 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 09:46:47.839478 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 09:46:47.839478 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 09:46:47.839478 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 09:46:47.839478 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:47.839478 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:47.839478 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:47.839478 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:47.839478 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:47.839478 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:47.839478 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:47.839478 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:47.839478 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:47.839478 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:47.839478 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:47.839478 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:47.840800 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:47.840800 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:47.840800 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 09:46:47.840800 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 09:46:47.840800 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 09:46:47.840800 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 09:46:47.840800 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 09:46:47.840800 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 09:46:47.840800 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 02 09:46:47.840800 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:46:47.843076 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 347/702] 10.222.0.64 () {56 vars in 1093 bytes} [Tue Aug 2 09:46:47 2022] GET /identity/v3/users/nova => generated 82 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:46:47.848681 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fcebb4-406f-4a6c-bc96-962639d823d6 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:47.908949 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:47.979509 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fcebb4-406f-4a6c-bc96-962639d823d6 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:48.014157 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489def6a-cabd-4416-8f31-0c7e3269b709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:48.014774 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489def6a-cabd-4416-8f31-0c7e3269b709 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:48.015057 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489def6a-cabd-4416-8f31-0c7e3269b709 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:48.015332 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489def6a-cabd-4416-8f31-0c7e3269b709 None admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:48.016228 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489def6a-cabd-4416-8f31-0c7e3269b709 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:48.016612 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489def6a-cabd-4416-8f31-0c7e3269b709 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:48.018071 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489def6a-cabd-4416-8f31-0c7e3269b709 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:48.041080 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 356/703] 10.222.0.64 () {56 vars in 1133 bytes} [Tue Aug 2 09:46:47 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:48.046230 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3513bdd4-7a14-468d-bcc6-a74b5ac6005b None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:48.172351 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3513bdd4-7a14-468d-bcc6-a74b5ac6005b None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:48.193592 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:48.209581 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ccfd91-4676-45e5-b47e-89ae57a3c555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:48.210269 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ccfd91-4676-45e5-b47e-89ae57a3c555 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:48.210604 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ccfd91-4676-45e5-b47e-89ae57a3c555 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:48.210889 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ccfd91-4676-45e5-b47e-89ae57a3c555 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:48.300411 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ccfd91-4676-45e5-b47e-89ae57a3c555 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:48.302251 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ccfd91-4676-45e5-b47e-89ae57a3c555 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:48.420853 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 348/704] 10.222.0.64 () {58 vars in 1282 bytes} [Tue Aug 2 09:46:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 09:46:48.429593 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489def6a-cabd-4416-8f31-0c7e3269b709 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:48.555216 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:48.559060 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489def6a-cabd-4416-8f31-0c7e3269b709 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:48.605845 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631eb23a-46ec-4f7a-aee2-d719cc167104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:48.606759 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631eb23a-46ec-4f7a-aee2-d719cc167104 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:48.607194 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631eb23a-46ec-4f7a-aee2-d719cc167104 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:48.607622 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631eb23a-46ec-4f7a-aee2-d719cc167104 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:48.619460 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631eb23a-46ec-4f7a-aee2-d719cc167104 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:48.625306 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631eb23a-46ec-4f7a-aee2-d719cc167104 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:48.645228 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-631eb23a-46ec-4f7a-aee2-d719cc167104 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:46:48.645228 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:48.645228 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:48.645228 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:48.645228 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:48.645228 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:48.645228 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:48.645228 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:48.645228 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:48.645228 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:48.645228 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:48.645228 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:48.645228 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 09:46:48.645228 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 09:46:48.645228 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 09:46:48.646394 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 09:46:48.646394 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:48.646394 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:48.646394 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:48.646394 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:48.646394 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:48.646394 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:48.646394 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:48.646394 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:48.646394 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:48.646394 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:48.646394 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:48.646394 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:48.646394 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:48.646394 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:48.646394 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:48.646394 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:48.647557 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 09:46:48.647557 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 09:46:48.647557 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:46:48.647557 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:46:48.649478 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 357/705] 10.222.0.64 () {56 vars in 1108 bytes} [Tue Aug 2 09:46:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:46:48.657924 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ccfd91-4676-45e5-b47e-89ae57a3c555 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:48.688708 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:48.786739 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ccfd91-4676-45e5-b47e-89ae57a3c555 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:48.822552 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e14bcad-6886-4178-aeb4-5f4fac2cd2bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:48.823179 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e14bcad-6886-4178-aeb4-5f4fac2cd2bc None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:48.823461 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e14bcad-6886-4178-aeb4-5f4fac2cd2bc None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:48.823733 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e14bcad-6886-4178-aeb4-5f4fac2cd2bc None admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:48.824464 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e14bcad-6886-4178-aeb4-5f4fac2cd2bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:48.824768 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e14bcad-6886-4178-aeb4-5f4fac2cd2bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:48.826138 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e14bcad-6886-4178-aeb4-5f4fac2cd2bc None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:48.839551 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 349/706] 10.222.0.64 () {56 vars in 1109 bytes} [Tue Aug 2 09:46:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:48.845364 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631eb23a-46ec-4f7a-aee2-d719cc167104 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:48.872803 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7676f5af-c71c-4276-9487-8a89da2c96a9 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:48.872803 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7676f5af-c71c-4276-9487-8a89da2c96a9 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:48.872803 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7676f5af-c71c-4276-9487-8a89da2c96a9 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:48.872803 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 350/707] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:46:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:46:48.876122 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859bf445-ae22-4975-9008-c65f6d28a5ef None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:48.876423 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859bf445-ae22-4975-9008-c65f6d28a5ef None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:48.876708 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859bf445-ae22-4975-9008-c65f6d28a5ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:48.900275 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-859bf445-ae22-4975-9008-c65f6d28a5ef None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:46:48.900275 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-859bf445-ae22-4975-9008-c65f6d28a5ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:46:48.982124 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631eb23a-46ec-4f7a-aee2-d719cc167104 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:49.023333 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a235d1-c699-4603-b002-4b86cfa64732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:49.024378 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a235d1-c699-4603-b002-4b86cfa64732 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:49.024877 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a235d1-c699-4603-b002-4b86cfa64732 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:49.025389 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a235d1-c699-4603-b002-4b86cfa64732 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:49.036658 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a235d1-c699-4603-b002-4b86cfa64732 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:49.039073 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a235d1-c699-4603-b002-4b86cfa64732 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:49.058543 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-10a235d1-c699-4603-b002-4b86cfa64732 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 09:46:49.058543 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:49.058543 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:49.058543 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:49.058543 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:49.058543 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:49.058543 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:49.058543 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:49.058543 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:49.058543 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:49.058543 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:49.058543 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:49.058543 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 09:46:49.058543 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 09:46:49.058543 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 09:46:49.059505 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 09:46:49.059505 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:49.059505 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:49.059505 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:49.059505 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:49.059505 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:49.059505 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:49.059505 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:49.059505 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:49.059505 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:49.059505 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:49.059505 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:49.059505 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:49.059505 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:49.059505 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:49.059505 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:49.059505 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:49.060434 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 09:46:49.060434 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 09:46:49.060434 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 09:46:49.060434 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 09:46:49.060434 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 09:46:49.060434 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 09:46:49.060434 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 09:46:49.060434 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:46:49.063152 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 351/708] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:46:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 09:46:49.064871 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 358/709] 10.222.0.64 () {56 vars in 1111 bytes} [Tue Aug 2 09:46:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:46:49.070870 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859bf445-ae22-4975-9008-c65f6d28a5ef None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:49.093088 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a235d1-c699-4603-b002-4b86cfa64732 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:49.104670 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:49.127414 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:49.187278 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859bf445-ae22-4975-9008-c65f6d28a5ef None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:49.226107 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1097167b-fe95-4594-83de-b27103d3cefd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:49.226738 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1097167b-fe95-4594-83de-b27103d3cefd None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:49.227121 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1097167b-fe95-4594-83de-b27103d3cefd None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:49.227404 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1097167b-fe95-4594-83de-b27103d3cefd None admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:49.228181 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1097167b-fe95-4594-83de-b27103d3cefd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:49.228520 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1097167b-fe95-4594-83de-b27103d3cefd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:49.230129 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1097167b-fe95-4594-83de-b27103d3cefd None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:49.230248 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a235d1-c699-4603-b002-4b86cfa64732 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:49.243356 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 352/710] 10.222.0.64 () {56 vars in 1148 bytes} [Tue Aug 2 09:46:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:49.244942 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3514742a-4956-4bf5-9830-30b5125a471d None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:49.245230 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3514742a-4956-4bf5-9830-30b5125a471d None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:49.245505 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3514742a-4956-4bf5-9830-30b5125a471d None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:49.246968 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 353/711] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:46:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:46:49.249788 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3514742a-4956-4bf5-9830-30b5125a471d None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:49.274043 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18d1ef2-b7e1-4b26-b28c-437742b5ecba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:49.274861 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18d1ef2-b7e1-4b26-b28c-437742b5ecba None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:49.275143 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18d1ef2-b7e1-4b26-b28c-437742b5ecba None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:49.275745 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18d1ef2-b7e1-4b26-b28c-437742b5ecba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:49.293781 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18d1ef2-b7e1-4b26-b28c-437742b5ecba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:49.295198 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18d1ef2-b7e1-4b26-b28c-437742b5ecba None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:49.313394 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-e18d1ef2-b7e1-4b26-b28c-437742b5ecba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:46:49.313394 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:49.313394 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:49.313394 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:49.313394 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:49.313394 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:49.313394 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:49.313394 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:49.313394 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:49.313394 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:49.313394 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:49.313394 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:49.313394 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 09:46:49.313394 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 09:46:49.313394 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 09:46:49.314371 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 09:46:49.314371 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:49.314371 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:49.314371 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:49.314371 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:49.314371 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:49.314371 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:49.314371 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:49.314371 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:49.314371 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:49.314371 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:49.314371 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:49.314371 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:49.314371 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:49.314371 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:49.314371 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:49.314371 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:49.315276 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 09:46:49.315276 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 09:46:49.315276 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:46:49.315276 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:46:49.316140 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 359/712] 10.222.0.64 () {56 vars in 1108 bytes} [Tue Aug 2 09:46:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 09:46:49.317144 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da30565-f3bc-4b73-810b-c24b8f83f0b3 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:49.317433 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da30565-f3bc-4b73-810b-c24b8f83f0b3 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:49.317710 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da30565-f3bc-4b73-810b-c24b8f83f0b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:49.344794 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-6da30565-f3bc-4b73-810b-c24b8f83f0b3 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:46:49.345618 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:49.348482 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-6da30565-f3bc-4b73-810b-c24b8f83f0b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:46:49.393463 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3514742a-4956-4bf5-9830-30b5125a471d None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:49.423855 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:49.436381 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 360/713] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:46:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 09:46:49.437245 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da30565-f3bc-4b73-810b-c24b8f83f0b3 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:49.441683 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53170a70-465e-4375-9f16-6c70eda4d7c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:49.442582 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53170a70-465e-4375-9f16-6c70eda4d7c9 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:49.443087 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53170a70-465e-4375-9f16-6c70eda4d7c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:49.443578 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53170a70-465e-4375-9f16-6c70eda4d7c9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:49.444604 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53170a70-465e-4375-9f16-6c70eda4d7c9 None admin] RBAC: Adding query filter params (role.id=1000cf318fa14915b8091599eaead5de, scope.project.id=390bd7c4f91c4465b0c7ab0f9e436c4d, user.id=99655e1f9ca24472a86cd26b0dbd1fc2) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:49.445126 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53170a70-465e-4375-9f16-6c70eda4d7c9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:49.446832 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53170a70-465e-4375-9f16-6c70eda4d7c9 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:49.454105 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 354/714] 10.222.0.64 () {56 vars in 1374 bytes} [Tue Aug 2 09:46:49 2022] GET /identity/v3/role_assignments?user.id=99655e1f9ca24472a86cd26b0dbd1fc2&scope.project.id=390bd7c4f91c4465b0c7ab0f9e436c4d&role.id=1000cf318fa14915b8091599eaead5de => generated 259 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:46:49.455883 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411ec06b-133f-4461-8933-936e16de371e None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:49.456418 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411ec06b-133f-4461-8933-936e16de371e None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:49.457222 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411ec06b-133f-4461-8933-936e16de371e None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:49.458793 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 355/715] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:46:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:46:49.459929 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411ec06b-133f-4461-8933-936e16de371e None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:49.568420 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da30565-f3bc-4b73-810b-c24b8f83f0b3 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:49.595606 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411ec06b-133f-4461-8933-936e16de371e None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:49.611862 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5f9047-a392-41ca-ae34-1042c9e1a5bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:49.613125 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5f9047-a392-41ca-ae34-1042c9e1a5bb None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:49.613620 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5f9047-a392-41ca-ae34-1042c9e1a5bb None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:49.614151 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5f9047-a392-41ca-ae34-1042c9e1a5bb None admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:49.615190 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5f9047-a392-41ca-ae34-1042c9e1a5bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:49.615964 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5f9047-a392-41ca-ae34-1042c9e1a5bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:49.617562 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5f9047-a392-41ca-ae34-1042c9e1a5bb None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:49.631899 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 361/716] 10.222.0.64 () {56 vars in 1109 bytes} [Tue Aug 2 09:46:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:49.633591 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd99d4c8-6393-49b5-b9d0-788344e3c429 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:49.634063 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd99d4c8-6393-49b5-b9d0-788344e3c429 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:49.634556 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd99d4c8-6393-49b5-b9d0-788344e3c429 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:49.641215 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d150279f-4f1d-4bf4-8e0e-c2fc1903a25d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:49.641882 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d150279f-4f1d-4bf4-8e0e-c2fc1903a25d None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:49.642173 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d150279f-4f1d-4bf4-8e0e-c2fc1903a25d None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:49.642454 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d150279f-4f1d-4bf4-8e0e-c2fc1903a25d None admin] PATH_INFO: `/v3/services/network` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:49.643098 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d150279f-4f1d-4bf4-8e0e-c2fc1903a25d None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:49.644529 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d150279f-4f1d-4bf4-8e0e-c2fc1903a25d None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:49.647426 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:49.657130 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-d150279f-4f1d-4bf4-8e0e-c2fc1903a25d None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 02 09:46:49.657130 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:49.657130 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 09:46:49.657130 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 09:46:49.657130 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 09:46:49.657130 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 09:46:49.657130 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 09:46:49.657130 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 09:46:49.657130 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 02 09:46:49.657130 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:46:49.657130 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 09:46:49.657130 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:46:49.657130 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:49.657130 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:49.657130 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:49.657130 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:49.657130 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:49.658154 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:49.658154 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:49.658154 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:49.658154 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:49.658154 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:49.658154 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:49.658154 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 09:46:49.658154 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 09:46:49.658154 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 09:46:49.658154 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 09:46:49.658154 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:49.658154 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:49.658154 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:49.658154 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:49.658154 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:49.658154 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:49.659027 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:49.659027 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:49.659027 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:49.659027 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:49.659027 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:49.659027 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:49.659027 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:49.659027 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:49.659027 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:49.659027 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:49.659027 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 09:46:49.659027 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 09:46:49.659027 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 02 09:46:49.659027 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:46:49.659792 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-fd99d4c8-6393-49b5-b9d0-788344e3c429 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:46:49.659792 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-fd99d4c8-6393-49b5-b9d0-788344e3c429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:46:49.662055 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 356/717] 10.222.0.64 () {56 vars in 1111 bytes} [Tue Aug 2 09:46:49 2022] GET /identity/v3/services/network => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:46:49.662918 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d150279f-4f1d-4bf4-8e0e-c2fc1903a25d None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:49.727253 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:49.752023 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 362/718] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:46:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 09:46:49.753489 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd99d4c8-6393-49b5-b9d0-788344e3c429 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:49.771103 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d150279f-4f1d-4bf4-8e0e-c2fc1903a25d None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:49.821476 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e15882-bd1a-44ae-94a6-5ee7ef4afed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:49.822518 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e15882-bd1a-44ae-94a6-5ee7ef4afed4 None admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:49.822908 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e15882-bd1a-44ae-94a6-5ee7ef4afed4 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:49.823702 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e15882-bd1a-44ae-94a6-5ee7ef4afed4 None admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:49.824797 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e15882-bd1a-44ae-94a6-5ee7ef4afed4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:49.827000 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e15882-bd1a-44ae-94a6-5ee7ef4afed4 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:49.861410 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 357/719] 10.222.0.64 () {60 vars in 1132 bytes} [Tue Aug 2 09:46:49 2022] POST /identity/v3/users => generated 254 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:46:49.862606 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e15882-bd1a-44ae-94a6-5ee7ef4afed4 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:49.892072 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd99d4c8-6393-49b5-b9d0-788344e3c429 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:49.936504 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a28b7ca-48d1-496d-b62a-99438d1eb94c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:49.937213 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a28b7ca-48d1-496d-b62a-99438d1eb94c None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:49.937507 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a28b7ca-48d1-496d-b62a-99438d1eb94c None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:49.938395 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a28b7ca-48d1-496d-b62a-99438d1eb94c None admin] PATH_INFO: `/v3/services` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:49.939478 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a28b7ca-48d1-496d-b62a-99438d1eb94c None admin] RBAC: Adding query filter params (name=network) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:49.939798 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a28b7ca-48d1-496d-b62a-99438d1eb94c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:49.941574 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a28b7ca-48d1-496d-b62a-99438d1eb94c None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:49.947331 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 363/720] 10.222.0.64 () {56 vars in 1112 bytes} [Tue Aug 2 09:46:49 2022] GET /identity/v3/services?name=network => generated 124 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:49.954939 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a28b7ca-48d1-496d-b62a-99438d1eb94c None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:49.988281 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:50.061920 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:50.085004 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e15882-bd1a-44ae-94a6-5ee7ef4afed4 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:50.133649 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2598357-26b0-4d64-9b1a-5573de3cae76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:50.134586 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2598357-26b0-4d64-9b1a-5573de3cae76 None admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:50.134876 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2598357-26b0-4d64-9b1a-5573de3cae76 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:50.135153 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2598357-26b0-4d64-9b1a-5573de3cae76 None admin] PATH_INFO: `/v3/services` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:50.136124 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2598357-26b0-4d64-9b1a-5573de3cae76 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:50.137603 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2598357-26b0-4d64-9b1a-5573de3cae76 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:50.138232 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a28b7ca-48d1-496d-b62a-99438d1eb94c None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:50.160988 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 358/721] 10.222.0.64 () {60 vars in 1141 bytes} [Tue Aug 2 09:46:49 2022] POST /identity/v3/services => generated 251 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:46:50.206743 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ead1c6-f40d-43b6-9f35-473a76b62838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:50.207794 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ead1c6-f40d-43b6-9f35-473a76b62838 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:50.208389 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ead1c6-f40d-43b6-9f35-473a76b62838 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:50.208900 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ead1c6-f40d-43b6-9f35-473a76b62838 None admin] PATH_INFO: `/v3/services` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:50.209925 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ead1c6-f40d-43b6-9f35-473a76b62838 None admin] RBAC: Adding query filter params (type=network) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:50.210549 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ead1c6-f40d-43b6-9f35-473a76b62838 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:50.212240 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ead1c6-f40d-43b6-9f35-473a76b62838 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:50.218630 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 364/722] 10.222.0.64 () {56 vars in 1112 bytes} [Tue Aug 2 09:46:49 2022] GET /identity/v3/services?type=network => generated 124 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:50.735211 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee831da-31b9-498e-880b-4f15dd8d2149 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:50.735582 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee831da-31b9-498e-880b-4f15dd8d2149 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:50.735831 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee831da-31b9-498e-880b-4f15dd8d2149 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:50.737527 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 359/723] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:46:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:46:50.742941 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d8691d-defd-43e2-9dda-4e9af4a2b5e2 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:50.743238 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d8691d-defd-43e2-9dda-4e9af4a2b5e2 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:50.743521 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d8691d-defd-43e2-9dda-4e9af4a2b5e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:50.769496 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-73d8691d-defd-43e2-9dda-4e9af4a2b5e2 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:46:50.770715 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-73d8691d-defd-43e2-9dda-4e9af4a2b5e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:46:50.865907 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 365/724] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:46:50 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 09:46:50.901223 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee831da-31b9-498e-880b-4f15dd8d2149 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:51.041736 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee831da-31b9-498e-880b-4f15dd8d2149 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:51.055780 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:51.080432 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b37df3b-fe6e-4ba0-8698-12621bd6db02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:51.084974 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b37df3b-fe6e-4ba0-8698-12621bd6db02 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:51.085268 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b37df3b-fe6e-4ba0-8698-12621bd6db02 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:51.085551 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b37df3b-fe6e-4ba0-8698-12621bd6db02 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:51.097324 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b37df3b-fe6e-4ba0-8698-12621bd6db02 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 09:46:51.106570 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-1b37df3b-fe6e-4ba0-8698-12621bd6db02 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 09:46:51.106570 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:51.106570 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:51.106570 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:51.106570 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:51.106570 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:51.106570 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:51.106570 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:51.106570 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:51.106570 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:51.106570 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:51.106570 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:51.106570 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 09:46:51.106570 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 09:46:51.106570 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 09:46:51.106570 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 09:46:51.107527 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 09:46:51.107527 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 09:46:51.107527 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 09:46:51.107527 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 09:46:51.107527 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:51.107527 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:51.107527 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:51.107527 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:51.107527 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:51.107527 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:51.107527 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:51.107527 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:51.107527 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:51.107527 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:51.107527 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:51.107527 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:51.108421 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:51.108421 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:51.108421 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:51.108421 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:51.108421 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 09:46:51.108421 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 09:46:51.108421 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 09:46:51.108421 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 09:46:51.108421 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 09:46:51.108421 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 09:46:51.108421 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 09:46:51.108421 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:46:51.108421 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 360/725] 10.222.0.64 () {56 vars in 1096 bytes} [Tue Aug 2 09:46:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:46:51.117363 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d8691d-defd-43e2-9dda-4e9af4a2b5e2 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:51.178067 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:51.263977 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d8691d-defd-43e2-9dda-4e9af4a2b5e2 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:51.322393 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02835f28-574c-4f2a-920b-d29c52bb8713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:51.323019 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02835f28-574c-4f2a-920b-d29c52bb8713 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:51.323303 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02835f28-574c-4f2a-920b-d29c52bb8713 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:51.323576 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02835f28-574c-4f2a-920b-d29c52bb8713 None admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:51.324328 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02835f28-574c-4f2a-920b-d29c52bb8713 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:51.324634 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02835f28-574c-4f2a-920b-d29c52bb8713 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:51.326075 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02835f28-574c-4f2a-920b-d29c52bb8713 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:51.336891 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 366/726] 10.222.0.64 () {56 vars in 1099 bytes} [Tue Aug 2 09:46:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:51.342458 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b37df3b-fe6e-4ba0-8698-12621bd6db02 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:51.585785 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b37df3b-fe6e-4ba0-8698-12621bd6db02 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:51.631000 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecec6a9-52e0-40fc-8328-6c9f57a8faef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:51.631654 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecec6a9-52e0-40fc-8328-6c9f57a8faef None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:51.631939 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecec6a9-52e0-40fc-8328-6c9f57a8faef None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:51.632212 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecec6a9-52e0-40fc-8328-6c9f57a8faef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:51.647919 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecec6a9-52e0-40fc-8328-6c9f57a8faef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:51.649421 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecec6a9-52e0-40fc-8328-6c9f57a8faef None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:51.665021 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-cecec6a9-52e0-40fc-8328-6c9f57a8faef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:46:51.665021 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:51.665021 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:51.665021 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:51.665021 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:51.665021 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:51.665021 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:51.665021 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:51.665021 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:51.665021 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:51.665021 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:51.665021 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:51.665021 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 09:46:51.665021 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 09:46:51.665021 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 09:46:51.666050 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 09:46:51.666050 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:51.666050 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:51.666050 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:51.666050 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:51.666050 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:51.666050 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:51.666050 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:51.666050 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:51.666050 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:51.666050 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:51.666050 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:51.666050 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:51.666050 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:51.666050 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:51.666050 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:51.666050 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:51.667008 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 09:46:51.667008 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 09:46:51.667008 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:46:51.667008 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:46:51.667008 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 361/727] 10.222.0.64 () {56 vars in 1108 bytes} [Tue Aug 2 09:46:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:46:51.671852 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02835f28-574c-4f2a-920b-d29c52bb8713 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:51.810682 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:51.819224 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02835f28-574c-4f2a-920b-d29c52bb8713 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:51.870426 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16304588-9781-47e7-9d34-e07290e93710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:51.870988 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16304588-9781-47e7-9d34-e07290e93710 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:51.871281 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16304588-9781-47e7-9d34-e07290e93710 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:51.871566 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16304588-9781-47e7-9d34-e07290e93710 None admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:51.872298 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16304588-9781-47e7-9d34-e07290e93710 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:51.872618 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16304588-9781-47e7-9d34-e07290e93710 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:51.874007 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16304588-9781-47e7-9d34-e07290e93710 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:51.897884 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 367/728] 10.222.0.64 () {56 vars in 1109 bytes} [Tue Aug 2 09:46:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:51.903467 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecec6a9-52e0-40fc-8328-6c9f57a8faef None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:51.935215 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6e06a1-020e-42da-b210-d3127b197565 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:51.935534 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6e06a1-020e-42da-b210-d3127b197565 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:51.935796 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6e06a1-020e-42da-b210-d3127b197565 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:51.937330 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 368/729] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:46:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 09:46:51.942632 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:51.943477 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24497a6-5f02-4da3-9537-a9eb9dd82192 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:51.943614 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24497a6-5f02-4da3-9537-a9eb9dd82192 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:51.943817 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24497a6-5f02-4da3-9537-a9eb9dd82192 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:51.977621 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-e24497a6-5f02-4da3-9537-a9eb9dd82192 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:46:51.979610 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-e24497a6-5f02-4da3-9537-a9eb9dd82192 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:46:52.098877 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecec6a9-52e0-40fc-8328-6c9f57a8faef None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:52.138857 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d28441-caa9-4d93-97e2-a4db511d3cc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:52.139867 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d28441-caa9-4d93-97e2-a4db511d3cc6 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:52.140449 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d28441-caa9-4d93-97e2-a4db511d3cc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:52.141515 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d28441-caa9-4d93-97e2-a4db511d3cc6 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:52.146272 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 369/730] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:46:51 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 09:46:52.150163 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d28441-caa9-4d93-97e2-a4db511d3cc6 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:52.151842 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d28441-caa9-4d93-97e2-a4db511d3cc6 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:52.164714 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-82d28441-caa9-4d93-97e2-a4db511d3cc6 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 02 09:46:52.164714 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:52.164714 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:52.164714 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:52.164714 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:52.164714 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:52.164714 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:52.164714 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:52.164714 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:52.164714 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:52.164714 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:52.164714 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:52.164714 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 09:46:52.164714 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 09:46:52.164714 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 09:46:52.164714 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 09:46:52.165777 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:52.165777 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:52.165777 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 09:46:52.165777 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 09:46:52.165777 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 09:46:52.165777 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 09:46:52.165777 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:52.165777 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:52.165777 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:52.165777 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:52.165777 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:52.165777 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:52.165777 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:52.165777 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:52.165777 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:52.165777 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:52.165777 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:52.165777 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:52.166741 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:52.166741 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:52.166741 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 09:46:52.166741 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 09:46:52.166741 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 09:46:52.166741 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 09:46:52.166741 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 09:46:52.166741 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 09:46:52.166741 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 02 09:46:52.166741 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:46:52.169550 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 362/731] 10.222.0.64 () {56 vars in 1096 bytes} [Tue Aug 2 09:46:51 2022] GET /identity/v3/users/swift => generated 83 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:46:52.175519 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24497a6-5f02-4da3-9537-a9eb9dd82192 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:52.185652 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d28441-caa9-4d93-97e2-a4db511d3cc6 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:52.300678 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:52.315442 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d28441-caa9-4d93-97e2-a4db511d3cc6 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:52.357753 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515c4ccd-dd4b-4c17-b8a5-3f4897f4756b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:52.358566 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515c4ccd-dd4b-4c17-b8a5-3f4897f4756b None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:52.359008 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515c4ccd-dd4b-4c17-b8a5-3f4897f4756b None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:52.359408 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515c4ccd-dd4b-4c17-b8a5-3f4897f4756b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:52.373727 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515c4ccd-dd4b-4c17-b8a5-3f4897f4756b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 09:46:52.379271 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-515c4ccd-dd4b-4c17-b8a5-3f4897f4756b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 09:46:52.379271 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:52.379271 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:52.379271 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:52.379271 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:52.379271 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:52.379271 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:52.379271 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:52.379271 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:52.379271 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:52.379271 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:52.379271 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:52.379271 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 09:46:52.379271 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 09:46:52.379271 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 09:46:52.379271 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 09:46:52.380371 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 09:46:52.380371 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 09:46:52.380371 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 09:46:52.380371 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 09:46:52.380371 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:52.380371 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:52.380371 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:52.380371 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:52.380371 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:52.380371 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:52.380371 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:52.380371 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:52.380371 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:52.380371 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:52.380371 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:52.380371 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:52.381260 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:52.381260 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:52.381260 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:52.381260 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:52.381260 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 09:46:52.381260 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 09:46:52.381260 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 09:46:52.381260 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 09:46:52.381260 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 09:46:52.381260 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 09:46:52.381260 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 09:46:52.381260 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:46:52.381928 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 363/732] 10.222.0.64 () {56 vars in 1096 bytes} [Tue Aug 2 09:46:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 09:46:52.387248 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515c4ccd-dd4b-4c17-b8a5-3f4897f4756b None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:52.466150 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:52.504058 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515c4ccd-dd4b-4c17-b8a5-3f4897f4756b None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:52.514252 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24497a6-5f02-4da3-9537-a9eb9dd82192 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:52.553013 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a90587b-5ab3-4f42-9368-fb6b2cb36043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:52.553913 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a90587b-5ab3-4f42-9368-fb6b2cb36043 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:52.554351 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a90587b-5ab3-4f42-9368-fb6b2cb36043 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:52.554779 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a90587b-5ab3-4f42-9368-fb6b2cb36043 None admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:52.555694 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a90587b-5ab3-4f42-9368-fb6b2cb36043 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:52.556164 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a90587b-5ab3-4f42-9368-fb6b2cb36043 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:52.557810 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a90587b-5ab3-4f42-9368-fb6b2cb36043 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:52.567003 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3f9ee5-a955-4e0b-a026-7513aa00b6b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:52.567475 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 364/733] 10.222.0.64 () {56 vars in 1099 bytes} [Tue Aug 2 09:46:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:52.567595 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3f9ee5-a955-4e0b-a026-7513aa00b6b0 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:52.572164 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3f9ee5-a955-4e0b-a026-7513aa00b6b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:52.572164 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3f9ee5-a955-4e0b-a026-7513aa00b6b0 None admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:52.572164 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3f9ee5-a955-4e0b-a026-7513aa00b6b0 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:52.572164 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3f9ee5-a955-4e0b-a026-7513aa00b6b0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:52.572164 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3f9ee5-a955-4e0b-a026-7513aa00b6b0 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:52.573759 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a90587b-5ab3-4f42-9368-fb6b2cb36043 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:52.588610 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:52.602671 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 370/734] 10.222.0.64 () {56 vars in 1135 bytes} [Tue Aug 2 09:46:52 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 427 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:52.613347 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3f9ee5-a955-4e0b-a026-7513aa00b6b0 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:52.733200 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a90587b-5ab3-4f42-9368-fb6b2cb36043 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:52.781317 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b4f1a6-f8e7-43c7-becb-44b1dea652de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:52.782540 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b4f1a6-f8e7-43c7-becb-44b1dea652de None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:52.783013 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b4f1a6-f8e7-43c7-becb-44b1dea652de None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:52.783776 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b4f1a6-f8e7-43c7-becb-44b1dea652de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:52.796693 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b4f1a6-f8e7-43c7-becb-44b1dea652de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:52.798823 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b4f1a6-f8e7-43c7-becb-44b1dea652de None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:52.804407 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3f9ee5-a955-4e0b-a026-7513aa00b6b0 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:52.815717 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-b9b4f1a6-f8e7-43c7-becb-44b1dea652de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:46:52.815717 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:52.815717 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:52.815717 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:52.815717 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:52.815717 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:52.815717 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:52.815717 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:52.815717 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:52.815717 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:52.815717 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:52.815717 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:52.815717 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 09:46:52.815717 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 09:46:52.815717 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 09:46:52.816938 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 09:46:52.816938 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:52.816938 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:52.816938 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:52.816938 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:52.816938 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:52.816938 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:52.816938 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:52.816938 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:52.816938 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:52.816938 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:52.816938 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:52.816938 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:52.816938 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:52.816938 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:52.816938 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:52.816938 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:52.817844 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 09:46:52.817844 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 09:46:52.817844 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:46:52.817844 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:46:52.821567 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 365/735] 10.222.0.64 () {56 vars in 1108 bytes} [Tue Aug 2 09:46:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:46:52.822768 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00026504-b820-4204-a307-6dab0e13b01f None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:52.823066 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00026504-b820-4204-a307-6dab0e13b01f None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:52.823352 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00026504-b820-4204-a307-6dab0e13b01f None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:52.824824 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 366/736] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:46:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:46:52.825871 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998a375d-3873-445e-9418-44ea548324f9 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:52.826140 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998a375d-3873-445e-9418-44ea548324f9 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:52.826423 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998a375d-3873-445e-9418-44ea548324f9 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:52.831643 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 367/737] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:46:52 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:46:52.831643 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998a375d-3873-445e-9418-44ea548324f9 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:52.856479 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2cf4f2-b9fb-4219-aa10-12210289a0b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:52.857227 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2cf4f2-b9fb-4219-aa10-12210289a0b7 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:52.857525 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2cf4f2-b9fb-4219-aa10-12210289a0b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:52.857788 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2cf4f2-b9fb-4219-aa10-12210289a0b7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:52.887780 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:52.908466 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:52.943757 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2cf4f2-b9fb-4219-aa10-12210289a0b7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:52.945257 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2cf4f2-b9fb-4219-aa10-12210289a0b7 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:52.955072 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998a375d-3873-445e-9418-44ea548324f9 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:52.997846 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519a6947-451c-4b13-97d6-d3513cacca8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:52.998668 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519a6947-451c-4b13-97d6-d3513cacca8d None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:52.999004 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519a6947-451c-4b13-97d6-d3513cacca8d None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:52.999415 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519a6947-451c-4b13-97d6-d3513cacca8d None admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:53.000169 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519a6947-451c-4b13-97d6-d3513cacca8d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:53.000599 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519a6947-451c-4b13-97d6-d3513cacca8d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:53.001898 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519a6947-451c-4b13-97d6-d3513cacca8d None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:53.014410 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 368/738] 10.222.0.64 () {56 vars in 1109 bytes} [Tue Aug 2 09:46:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:53.015724 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314f7c00-c835-4913-b5e3-12a238f8ccde None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:53.016011 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314f7c00-c835-4913-b5e3-12a238f8ccde None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:53.016309 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314f7c00-c835-4913-b5e3-12a238f8ccde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:53.034272 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-314f7c00-c835-4913-b5e3-12a238f8ccde None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:46:53.035474 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-314f7c00-c835-4913-b5e3-12a238f8ccde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:46:53.065562 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 371/739] 10.222.0.64 () {58 vars in 1282 bytes} [Tue Aug 2 09:46:52 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 09:46:53.067061 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60609ce2-ec8c-45d0-963e-20916beb6088 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:53.067354 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60609ce2-ec8c-45d0-963e-20916beb6088 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:53.067633 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60609ce2-ec8c-45d0-963e-20916beb6088 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:53.084985 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-60609ce2-ec8c-45d0-963e-20916beb6088 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:46:53.086193 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-60609ce2-ec8c-45d0-963e-20916beb6088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:46:53.104587 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 369/740] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:46:53 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 09:46:53.105962 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cbd72c-7318-4b3d-8798-1e3582198c69 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:53.106275 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cbd72c-7318-4b3d-8798-1e3582198c69 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:53.106532 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cbd72c-7318-4b3d-8798-1e3582198c69 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:53.107824 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 370/741] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:46:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:46:53.108538 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cbd72c-7318-4b3d-8798-1e3582198c69 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:53.142905 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:53.162816 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:53.167203 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 372/742] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:46:53 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 09:46:53.168134 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60609ce2-ec8c-45d0-963e-20916beb6088 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:53.222140 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cbd72c-7318-4b3d-8798-1e3582198c69 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:53.259516 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea4cf34-1cf5-4c1e-8502-bfe9d9d2a678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:53.260068 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea4cf34-1cf5-4c1e-8502-bfe9d9d2a678 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:53.260381 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea4cf34-1cf5-4c1e-8502-bfe9d9d2a678 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:53.260656 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea4cf34-1cf5-4c1e-8502-bfe9d9d2a678 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:53.268606 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea4cf34-1cf5-4c1e-8502-bfe9d9d2a678 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:53.269793 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60609ce2-ec8c-45d0-963e-20916beb6088 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:53.270045 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea4cf34-1cf5-4c1e-8502-bfe9d9d2a678 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:53.282600 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-8ea4cf34-1cf5-4c1e-8502-bfe9d9d2a678 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 02 09:46:53.282600 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:53.282600 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:53.282600 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:53.282600 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:53.282600 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:53.282600 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:53.282600 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:53.282600 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:53.282600 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:53.282600 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:53.282600 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:53.282600 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 09:46:53.282600 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 09:46:53.282600 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 09:46:53.282600 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 09:46:53.283803 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:53.283803 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:53.283803 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 09:46:53.283803 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 09:46:53.283803 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 09:46:53.283803 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 09:46:53.283803 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:53.283803 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:53.283803 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:53.283803 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:53.283803 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:53.283803 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:53.283803 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:53.283803 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:53.283803 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:53.283803 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:53.283803 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:53.283803 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:53.284824 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:53.284824 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:53.284824 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 09:46:53.284824 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 09:46:53.284824 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 09:46:53.284824 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 09:46:53.284824 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 09:46:53.284824 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 09:46:53.284824 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 02 09:46:53.284824 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:46:53.284824 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 371/743] 10.222.0.64 () {56 vars in 1093 bytes} [Tue Aug 2 09:46:53 2022] GET /identity/v3/users/nova => generated 82 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:46:53.285431 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6fef5c-9f9a-4dbc-9512-8c51140c7ee1 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:53.285431 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6fef5c-9f9a-4dbc-9512-8c51140c7ee1 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:53.285557 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6fef5c-9f9a-4dbc-9512-8c51140c7ee1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:53.304137 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-8e6fef5c-9f9a-4dbc-9512-8c51140c7ee1 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:46:53.305392 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-8e6fef5c-9f9a-4dbc-9512-8c51140c7ee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:46:53.311633 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5af3ef0-0370-4bf5-b8c4-2ce4f12d3432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:53.312296 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5af3ef0-0370-4bf5-b8c4-2ce4f12d3432 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:53.312720 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5af3ef0-0370-4bf5-b8c4-2ce4f12d3432 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:53.313418 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5af3ef0-0370-4bf5-b8c4-2ce4f12d3432 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:53.323501 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5af3ef0-0370-4bf5-b8c4-2ce4f12d3432 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:53.324985 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5af3ef0-0370-4bf5-b8c4-2ce4f12d3432 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:53.340290 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-d5af3ef0-0370-4bf5-b8c4-2ce4f12d3432 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:46:53.340290 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:53.340290 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:53.340290 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:53.340290 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:53.340290 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:53.340290 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:53.340290 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:53.340290 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:53.340290 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:53.340290 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:53.340290 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:53.340290 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 09:46:53.340290 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 09:46:53.340290 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 09:46:53.341324 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 09:46:53.341324 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:53.341324 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:53.341324 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:53.341324 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:53.341324 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:53.341324 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:53.341324 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:53.341324 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:53.341324 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:53.341324 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:53.341324 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:53.341324 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:53.341324 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:53.341324 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:53.341324 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:53.341324 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:53.342392 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 09:46:53.342392 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 09:46:53.342392 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:46:53.342392 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:46:53.342392 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 373/744] 10.222.0.64 () {56 vars in 1108 bytes} [Tue Aug 2 09:46:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:46:53.342392 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5af3ef0-0370-4bf5-b8c4-2ce4f12d3432 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:53.376224 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:53.378586 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 372/745] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:46:53 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 09:46:53.379589 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6fef5c-9f9a-4dbc-9512-8c51140c7ee1 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:53.427956 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:53.455419 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5af3ef0-0370-4bf5-b8c4-2ce4f12d3432 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:53.488153 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6fef5c-9f9a-4dbc-9512-8c51140c7ee1 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:53.502653 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf2b5e1-f7b8-4051-ab9a-7fab9db9fb76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:53.503515 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf2b5e1-f7b8-4051-ab9a-7fab9db9fb76 None admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:53.503967 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf2b5e1-f7b8-4051-ab9a-7fab9db9fb76 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:53.504457 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf2b5e1-f7b8-4051-ab9a-7fab9db9fb76 None admin] PATH_INFO: `/v3/services` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:53.505611 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf2b5e1-f7b8-4051-ab9a-7fab9db9fb76 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:53.507992 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf2b5e1-f7b8-4051-ab9a-7fab9db9fb76 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:53.527533 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 374/746] 10.222.0.64 () {60 vars in 1141 bytes} [Tue Aug 2 09:46:53 2022] POST /identity/v3/services => generated 240 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:46:53.528294 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf2b5e1-f7b8-4051-ab9a-7fab9db9fb76 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:53.543537 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0530366-98ce-46d3-8f83-6f2fcf4fd9c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:53.544529 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0530366-98ce-46d3-8f83-6f2fcf4fd9c3 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:53.544816 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0530366-98ce-46d3-8f83-6f2fcf4fd9c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:53.545094 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0530366-98ce-46d3-8f83-6f2fcf4fd9c3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:53.558169 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0530366-98ce-46d3-8f83-6f2fcf4fd9c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 09:46:53.563382 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-e0530366-98ce-46d3-8f83-6f2fcf4fd9c3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 09:46:53.563382 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:53.563382 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:53.563382 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:53.563382 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:53.563382 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:53.563382 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:53.563382 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:53.563382 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:53.563382 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:53.563382 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:53.563382 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:53.563382 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 09:46:53.563382 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 09:46:53.563382 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 09:46:53.563382 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 09:46:53.564720 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 09:46:53.564720 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 09:46:53.564720 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 09:46:53.564720 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 09:46:53.564720 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:53.564720 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:53.564720 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:53.564720 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:53.564720 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:53.564720 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:53.564720 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:53.564720 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:53.564720 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:53.564720 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:53.564720 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:53.564720 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:53.566299 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:53.566299 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:53.566299 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:53.566299 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:53.566299 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 09:46:53.566299 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 09:46:53.566299 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 09:46:53.566299 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 09:46:53.566299 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 09:46:53.566299 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 09:46:53.566299 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 09:46:53.566299 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:46:53.566299 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 373/747] 10.222.0.64 () {56 vars in 1102 bytes} [Tue Aug 2 09:46:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:46:53.566299 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0530366-98ce-46d3-8f83-6f2fcf4fd9c3 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:53.660923 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:53.674885 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf2b5e1-f7b8-4051-ab9a-7fab9db9fb76 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:53.710741 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:53.720526 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0530366-98ce-46d3-8f83-6f2fcf4fd9c3 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:53.723881 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e098261f-b7f9-4692-87e4-71f630736434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:53.724793 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e098261f-b7f9-4692-87e4-71f630736434 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:53.725081 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e098261f-b7f9-4692-87e4-71f630736434 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:53.725354 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e098261f-b7f9-4692-87e4-71f630736434 None admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:53.726168 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e098261f-b7f9-4692-87e4-71f630736434 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:53.726477 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e098261f-b7f9-4692-87e4-71f630736434 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:53.727920 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e098261f-b7f9-4692-87e4-71f630736434 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:53.746302 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 375/748] 10.222.0.64 () {56 vars in 1133 bytes} [Tue Aug 2 09:46:53 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:53.747150 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e098261f-b7f9-4692-87e4-71f630736434 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:53.763924 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd492af9-4e87-4102-b7ab-f3fdf84f3d06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:53.764694 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd492af9-4e87-4102-b7ab-f3fdf84f3d06 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:53.765032 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd492af9-4e87-4102-b7ab-f3fdf84f3d06 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:53.765316 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd492af9-4e87-4102-b7ab-f3fdf84f3d06 None admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:53.766594 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd492af9-4e87-4102-b7ab-f3fdf84f3d06 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:53.766905 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd492af9-4e87-4102-b7ab-f3fdf84f3d06 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:53.768346 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd492af9-4e87-4102-b7ab-f3fdf84f3d06 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:53.780681 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 374/749] 10.222.0.64 () {56 vars in 1109 bytes} [Tue Aug 2 09:46:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:53.781575 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd492af9-4e87-4102-b7ab-f3fdf84f3d06 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:53.866010 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e098261f-b7f9-4692-87e4-71f630736434 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:53.896796 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd492af9-4e87-4102-b7ab-f3fdf84f3d06 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:53.910040 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e928cac6-611b-4841-81c1-115fa5aceb1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:53.910683 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e928cac6-611b-4841-81c1-115fa5aceb1b None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:53.910967 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e928cac6-611b-4841-81c1-115fa5aceb1b None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:53.911232 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e928cac6-611b-4841-81c1-115fa5aceb1b None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:53.911866 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e928cac6-611b-4841-81c1-115fa5aceb1b None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:53.913272 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e928cac6-611b-4841-81c1-115fa5aceb1b None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:53.924276 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-e928cac6-611b-4841-81c1-115fa5aceb1b None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 02 09:46:53.924276 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:53.924276 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 09:46:53.924276 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 09:46:53.924276 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 09:46:53.924276 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 09:46:53.924276 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 09:46:53.924276 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 09:46:53.924276 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 02 09:46:53.924276 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:46:53.924276 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 09:46:53.924276 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:46:53.924276 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:53.924276 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:53.924276 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:53.924276 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:53.925471 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:53.925471 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:53.925471 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:53.925471 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:53.925471 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:53.925471 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:53.925471 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:53.925471 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 09:46:53.925471 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 09:46:53.925471 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 09:46:53.925471 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 09:46:53.925471 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:53.925471 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:53.925471 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:53.925471 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:53.925471 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:53.926366 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:53.926366 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:53.926366 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:53.926366 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:53.926366 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:53.926366 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:53.926366 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:53.926366 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:53.926366 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:53.926366 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:53.926366 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:53.926366 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 09:46:53.926366 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 09:46:53.926366 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 02 09:46:53.926366 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:46:53.926366 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 376/750] 10.222.0.64 () {56 vars in 1129 bytes} [Tue Aug 2 09:46:53 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:46:53.927351 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e928cac6-611b-4841-81c1-115fa5aceb1b None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:53.939333 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0271b3ab-7245-46d3-8812-7c7b5adcc7bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:53.940434 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0271b3ab-7245-46d3-8812-7c7b5adcc7bc None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:53.940791 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0271b3ab-7245-46d3-8812-7c7b5adcc7bc None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:53.941076 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0271b3ab-7245-46d3-8812-7c7b5adcc7bc None admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:53.943409 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0271b3ab-7245-46d3-8812-7c7b5adcc7bc None admin] RBAC: Adding query filter params (name=service) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:53.943409 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0271b3ab-7245-46d3-8812-7c7b5adcc7bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:53.944416 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0271b3ab-7245-46d3-8812-7c7b5adcc7bc None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:53.946297 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:53.954957 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 375/751] 10.222.0.64 () {56 vars in 1103 bytes} [Tue Aug 2 09:46:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:53.955701 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0271b3ab-7245-46d3-8812-7c7b5adcc7bc None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:53.979371 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:54.060953 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e928cac6-611b-4841-81c1-115fa5aceb1b None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:54.072107 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0271b3ab-7245-46d3-8812-7c7b5adcc7bc None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:54.106441 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dfd7b8-8cfe-4d5c-b5cb-11147b11e454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:54.107109 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dfd7b8-8cfe-4d5c-b5cb-11147b11e454 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:54.107395 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dfd7b8-8cfe-4d5c-b5cb-11147b11e454 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:54.107671 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dfd7b8-8cfe-4d5c-b5cb-11147b11e454 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:54.109408 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5808a16b-2596-48f4-b3ab-1789c72f3805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:54.110055 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5808a16b-2596-48f4-b3ab-1789c72f3805 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:54.110343 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5808a16b-2596-48f4-b3ab-1789c72f3805 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:54.110887 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5808a16b-2596-48f4-b3ab-1789c72f3805 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:54.118024 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5808a16b-2596-48f4-b3ab-1789c72f3805 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:54.119466 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dfd7b8-8cfe-4d5c-b5cb-11147b11e454 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:54.119593 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5808a16b-2596-48f4-b3ab-1789c72f3805 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:54.120942 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dfd7b8-8cfe-4d5c-b5cb-11147b11e454 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:54.131431 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-5808a16b-2596-48f4-b3ab-1789c72f3805 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 09:46:54.131431 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:54.131431 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:54.131431 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:54.131431 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:54.131431 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:54.131431 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:54.131431 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:54.131431 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:54.131431 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:54.131431 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:54.131431 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:54.131431 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 09:46:54.131431 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 09:46:54.131431 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 09:46:54.133217 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 09:46:54.133217 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:54.133217 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:54.133217 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:54.133217 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:54.133217 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:54.133217 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:54.133217 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:54.133217 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:54.133217 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:54.133217 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:54.133217 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:54.133217 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:54.133217 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:54.133217 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:54.133217 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:54.133217 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:54.134920 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 09:46:54.134920 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 09:46:54.134920 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 09:46:54.134920 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 09:46:54.134920 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 09:46:54.134920 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 09:46:54.134920 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 09:46:54.134920 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:46:54.136911 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-e2dfd7b8-8cfe-4d5c-b5cb-11147b11e454 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:46:54.136911 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:54.136911 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:54.136911 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:54.136911 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:54.136911 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:54.136911 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:54.136911 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:54.136911 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:54.136911 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:54.136911 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:54.136911 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:54.136911 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 09:46:54.136911 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 09:46:54.136911 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 09:46:54.148312 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 09:46:54.148312 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:54.148312 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:54.148312 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:54.148312 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:54.148312 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:54.148312 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:54.148312 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:54.148312 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:54.148312 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:54.148312 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:54.148312 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:54.148312 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:54.148312 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:54.148312 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:54.148312 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:54.148312 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:54.154598 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 09:46:54.154598 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 09:46:54.154598 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:46:54.154598 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:46:54.155441 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 376/752] 10.222.0.64 () {56 vars in 1111 bytes} [Tue Aug 2 09:46:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:46:54.155441 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5808a16b-2596-48f4-b3ab-1789c72f3805 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:54.155916 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 377/753] 10.222.0.64 () {56 vars in 1108 bytes} [Tue Aug 2 09:46:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:46:54.155916 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dfd7b8-8cfe-4d5c-b5cb-11147b11e454 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:54.236379 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:54.257753 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:54.283005 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dfd7b8-8cfe-4d5c-b5cb-11147b11e454 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:54.286682 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5808a16b-2596-48f4-b3ab-1789c72f3805 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:54.321393 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b2cb2d-41a1-4c40-b30e-458316c79eb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:54.321977 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b2cb2d-41a1-4c40-b30e-458316c79eb8 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:54.322255 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b2cb2d-41a1-4c40-b30e-458316c79eb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:54.322521 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b2cb2d-41a1-4c40-b30e-458316c79eb8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:54.324297 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b547c19f-8712-419d-8910-1b3de61ebd62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:54.325102 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b547c19f-8712-419d-8910-1b3de61ebd62 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:54.325554 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b547c19f-8712-419d-8910-1b3de61ebd62 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:54.325838 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b547c19f-8712-419d-8910-1b3de61ebd62 None admin] PATH_INFO: `/v3/services` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:54.326532 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b547c19f-8712-419d-8910-1b3de61ebd62 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:54.326841 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b547c19f-8712-419d-8910-1b3de61ebd62 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:54.328778 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b547c19f-8712-419d-8910-1b3de61ebd62 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:54.333942 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 377/754] 10.222.0.64 () {56 vars in 1124 bytes} [Tue Aug 2 09:46:54 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:54.334616 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b547c19f-8712-419d-8910-1b3de61ebd62 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:54.341447 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b2cb2d-41a1-4c40-b30e-458316c79eb8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:54.342921 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b2cb2d-41a1-4c40-b30e-458316c79eb8 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:54.359165 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-c2b2cb2d-41a1-4c40-b30e-458316c79eb8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:46:54.359165 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:54.359165 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:54.359165 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:54.359165 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:54.359165 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:54.359165 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:54.359165 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:54.359165 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:54.359165 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:54.359165 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:54.359165 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:54.359165 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 09:46:54.359165 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 09:46:54.359165 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 09:46:54.360324 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 09:46:54.360324 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:54.360324 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:54.360324 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:54.360324 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:54.360324 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:54.360324 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:54.360324 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:54.360324 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:54.360324 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:54.360324 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:54.360324 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:54.360324 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:54.360324 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:54.360324 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:54.360324 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:54.360324 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:54.361269 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 09:46:54.361269 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 09:46:54.361269 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:46:54.361269 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:46:54.361269 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 378/755] 10.222.0.64 () {56 vars in 1108 bytes} [Tue Aug 2 09:46:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:46:54.361269 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b2cb2d-41a1-4c40-b30e-458316c79eb8 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:54.468361 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b2cb2d-41a1-4c40-b30e-458316c79eb8 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:54.469547 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b547c19f-8712-419d-8910-1b3de61ebd62 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:54.489493 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:54.509799 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1590a9-13c6-4cf6-b65d-ac85bd0a3704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:54.510529 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1590a9-13c6-4cf6-b65d-ac85bd0a3704 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:54.510798 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea477bbc-da06-4bfd-bbed-e947b3120bb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:54.510949 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1590a9-13c6-4cf6-b65d-ac85bd0a3704 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:54.511296 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1590a9-13c6-4cf6-b65d-ac85bd0a3704 None admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:54.511596 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea477bbc-da06-4bfd-bbed-e947b3120bb6 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:54.511960 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea477bbc-da06-4bfd-bbed-e947b3120bb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:54.512285 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1590a9-13c6-4cf6-b65d-ac85bd0a3704 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:54.512384 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea477bbc-da06-4bfd-bbed-e947b3120bb6 None admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:54.512725 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1590a9-13c6-4cf6-b65d-ac85bd0a3704 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:54.513182 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea477bbc-da06-4bfd-bbed-e947b3120bb6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:54.513549 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea477bbc-da06-4bfd-bbed-e947b3120bb6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:54.514357 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1590a9-13c6-4cf6-b65d-ac85bd0a3704 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:54.515029 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea477bbc-da06-4bfd-bbed-e947b3120bb6 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:54.536548 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 378/756] 10.222.0.64 () {56 vars in 1148 bytes} [Tue Aug 2 09:46:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:54.536745 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 379/757] 10.222.0.64 () {56 vars in 1109 bytes} [Tue Aug 2 09:46:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:54.536835 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1590a9-13c6-4cf6-b65d-ac85bd0a3704 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:54.536909 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea477bbc-da06-4bfd-bbed-e947b3120bb6 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:54.539703 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:54.642025 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea477bbc-da06-4bfd-bbed-e947b3120bb6 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:54.647433 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1590a9-13c6-4cf6-b65d-ac85bd0a3704 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:54.683960 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee8d746-9f51-4aaf-aff8-5a31925a8c6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:54.684604 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee8d746-9f51-4aaf-aff8-5a31925a8c6d None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:54.684888 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee8d746-9f51-4aaf-aff8-5a31925a8c6d None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:54.685057 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c59fb2-49ce-4d58-8d48-9b03da838bf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:54.685168 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee8d746-9f51-4aaf-aff8-5a31925a8c6d None admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:54.685676 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c59fb2-49ce-4d58-8d48-9b03da838bf6 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:54.685847 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee8d746-9f51-4aaf-aff8-5a31925a8c6d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:54.686008 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c59fb2-49ce-4d58-8d48-9b03da838bf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:54.686177 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee8d746-9f51-4aaf-aff8-5a31925a8c6d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:54.686269 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c59fb2-49ce-4d58-8d48-9b03da838bf6 None admin] PATH_INFO: `/v3/services` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:54.686952 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c59fb2-49ce-4d58-8d48-9b03da838bf6 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:54.687255 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c59fb2-49ce-4d58-8d48-9b03da838bf6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:54.688126 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee8d746-9f51-4aaf-aff8-5a31925a8c6d None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:54.689169 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c59fb2-49ce-4d58-8d48-9b03da838bf6 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:54.700610 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 379/758] 10.222.0.64 () {56 vars in 1124 bytes} [Tue Aug 2 09:46:54 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:54.700610 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c59fb2-49ce-4d58-8d48-9b03da838bf6 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:54.709691 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 380/759] 10.222.0.64 () {56 vars in 1109 bytes} [Tue Aug 2 09:46:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:54.709691 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee8d746-9f51-4aaf-aff8-5a31925a8c6d None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:54.744747 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:54.774209 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:54.812190 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c59fb2-49ce-4d58-8d48-9b03da838bf6 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:54.828492 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee8d746-9f51-4aaf-aff8-5a31925a8c6d None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:54.853710 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad37eb71-a2e5-47b0-97e7-f8a36faebf23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:54.854458 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad37eb71-a2e5-47b0-97e7-f8a36faebf23 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:54.854810 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad37eb71-a2e5-47b0-97e7-f8a36faebf23 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:54.855155 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad37eb71-a2e5-47b0-97e7-f8a36faebf23 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:54.856034 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad37eb71-a2e5-47b0-97e7-f8a36faebf23 None admin] RBAC: Adding query filter params (role.id=1000cf318fa14915b8091599eaead5de, scope.project.id=390bd7c4f91c4465b0c7ab0f9e436c4d, user.id=6a371e3df3764a4cbb1b8368aa74bd65) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:54.856852 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad37eb71-a2e5-47b0-97e7-f8a36faebf23 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:54.857961 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad37eb71-a2e5-47b0-97e7-f8a36faebf23 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:54.872285 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 380/760] 10.222.0.64 () {56 vars in 1374 bytes} [Tue Aug 2 09:46:54 2022] GET /identity/v3/role_assignments?user.id=6a371e3df3764a4cbb1b8368aa74bd65&scope.project.id=390bd7c4f91c4465b0c7ab0f9e436c4d&role.id=1000cf318fa14915b8091599eaead5de => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:54.872285 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad37eb71-a2e5-47b0-97e7-f8a36faebf23 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:54.877127 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063d2a29-eaed-4fd8-bd97-e37bdcbf67bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:54.877794 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063d2a29-eaed-4fd8-bd97-e37bdcbf67bc None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:54.878083 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063d2a29-eaed-4fd8-bd97-e37bdcbf67bc None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:54.878360 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063d2a29-eaed-4fd8-bd97-e37bdcbf67bc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:54.888997 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063d2a29-eaed-4fd8-bd97-e37bdcbf67bc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:54.890535 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063d2a29-eaed-4fd8-bd97-e37bdcbf67bc None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:54.903668 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-063d2a29-eaed-4fd8-bd97-e37bdcbf67bc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 09:46:54.903668 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:54.903668 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:54.903668 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:54.903668 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:54.903668 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:54.903668 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:54.903668 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:54.903668 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:54.903668 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:54.903668 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:54.903668 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:54.903668 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 09:46:54.903668 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 09:46:54.903668 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 09:46:54.904664 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 09:46:54.904664 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:54.904664 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:54.904664 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:54.904664 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:54.904664 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:54.904664 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:54.904664 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:54.904664 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:54.904664 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:54.904664 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:54.904664 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:54.904664 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:54.904664 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:54.904664 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:54.904664 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:54.904664 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:54.905591 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 09:46:54.905591 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 09:46:54.905591 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 09:46:54.905591 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 09:46:54.905591 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 09:46:54.905591 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 09:46:54.905591 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 09:46:54.905591 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:46:54.905591 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 381/761] 10.222.0.64 () {56 vars in 1111 bytes} [Tue Aug 2 09:46:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:46:54.905591 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063d2a29-eaed-4fd8-bd97-e37bdcbf67bc None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:55.020218 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad37eb71-a2e5-47b0-97e7-f8a36faebf23 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:55.024090 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:55.034166 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063d2a29-eaed-4fd8-bd97-e37bdcbf67bc None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:55.064865 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:55.073288 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178cbbfe-5d7c-4685-abce-e5a8c3edef25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:55.074244 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178cbbfe-5d7c-4685-abce-e5a8c3edef25 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:55.074745 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178cbbfe-5d7c-4685-abce-e5a8c3edef25 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:55.075122 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178cbbfe-5d7c-4685-abce-e5a8c3edef25 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:55.076013 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178cbbfe-5d7c-4685-abce-e5a8c3edef25 None admin] RBAC: Adding query filter params (interface=public, service_id=c5bacae0322b4d9c81542c636e7645e9, region_id=RegionOne) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:55.076458 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178cbbfe-5d7c-4685-abce-e5a8c3edef25 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:55.077961 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178cbbfe-5d7c-4685-abce-e5a8c3edef25 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:55.079887 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09056a11-a438-4066-a4d6-92702a14b61b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:55.080756 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09056a11-a438-4066-a4d6-92702a14b61b None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:55.081215 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09056a11-a438-4066-a4d6-92702a14b61b None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:55.081638 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09056a11-a438-4066-a4d6-92702a14b61b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:55.087534 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 381/762] 10.222.0.64 () {56 vars in 1251 bytes} [Tue Aug 2 09:46:54 2022] GET /identity/v3/endpoints?service_id=c5bacae0322b4d9c81542c636e7645e9&interface=public®ion_id=RegionOne => generated 194 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:55.092585 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178cbbfe-5d7c-4685-abce-e5a8c3edef25 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:55.095056 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09056a11-a438-4066-a4d6-92702a14b61b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:55.096748 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09056a11-a438-4066-a4d6-92702a14b61b None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:55.111061 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-09056a11-a438-4066-a4d6-92702a14b61b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 02 09:46:55.111061 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:55.111061 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:55.111061 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:55.111061 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:55.111061 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:55.111061 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:55.111061 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:55.111061 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:55.111061 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:55.111061 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:55.111061 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:55.111061 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 09:46:55.111061 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 09:46:55.111061 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 09:46:55.111061 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 09:46:55.112283 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:55.112283 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:55.112283 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 09:46:55.112283 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 09:46:55.112283 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 09:46:55.112283 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 09:46:55.112283 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:55.112283 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:55.112283 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:55.112283 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:55.112283 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:55.112283 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:55.112283 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:55.112283 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:55.112283 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:55.112283 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:55.112283 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:55.112283 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:55.113488 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:55.113488 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:55.113488 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 09:46:55.113488 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 09:46:55.113488 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 09:46:55.113488 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 09:46:55.113488 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 09:46:55.113488 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 09:46:55.113488 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 02 09:46:55.113488 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:46:55.120143 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 382/763] 10.222.0.64 () {56 vars in 1117 bytes} [Tue Aug 2 09:46:54 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:46:55.121342 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09056a11-a438-4066-a4d6-92702a14b61b None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:55.215693 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178cbbfe-5d7c-4685-abce-e5a8c3edef25 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:55.258239 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09056a11-a438-4066-a4d6-92702a14b61b None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:55.262945 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da6edc2-033a-49d3-a40a-5e34e096fb8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:55.263784 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da6edc2-033a-49d3-a40a-5e34e096fb8f None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:55.264171 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da6edc2-033a-49d3-a40a-5e34e096fb8f None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:55.264569 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da6edc2-033a-49d3-a40a-5e34e096fb8f None admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:55.265480 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da6edc2-033a-49d3-a40a-5e34e096fb8f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:55.265947 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da6edc2-033a-49d3-a40a-5e34e096fb8f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:55.267509 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da6edc2-033a-49d3-a40a-5e34e096fb8f None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:55.285391 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 382/764] 10.222.0.64 () {56 vars in 1148 bytes} [Tue Aug 2 09:46:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:55.286872 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da6edc2-033a-49d3-a40a-5e34e096fb8f None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:55.309187 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4b9a9a-bf3a-41ee-8259-e3c3c6611ef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:55.310205 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4b9a9a-bf3a-41ee-8259-e3c3c6611ef2 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:55.310662 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4b9a9a-bf3a-41ee-8259-e3c3c6611ef2 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:55.311083 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4b9a9a-bf3a-41ee-8259-e3c3c6611ef2 None admin] PATH_INFO: `/v3/services` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:55.311898 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4b9a9a-bf3a-41ee-8259-e3c3c6611ef2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:55.314220 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4b9a9a-bf3a-41ee-8259-e3c3c6611ef2 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:55.324237 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 383/765] 10.222.0.64 () {56 vars in 1087 bytes} [Tue Aug 2 09:46:55 2022] GET /identity/v3/services => generated 772 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:55.326038 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4b9a9a-bf3a-41ee-8259-e3c3c6611ef2 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:55.376405 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:55.396678 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:55.512340 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da6edc2-033a-49d3-a40a-5e34e096fb8f None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:55.541131 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4b9a9a-bf3a-41ee-8259-e3c3c6611ef2 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:55.557497 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b37b494-5029-4a32-9ed6-3d5853345f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:55.558130 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b37b494-5029-4a32-9ed6-3d5853345f9e None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:55.558412 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b37b494-5029-4a32-9ed6-3d5853345f9e None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:55.558684 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b37b494-5029-4a32-9ed6-3d5853345f9e None admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:55.559618 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b37b494-5029-4a32-9ed6-3d5853345f9e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:55.559931 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b37b494-5029-4a32-9ed6-3d5853345f9e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:55.562569 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b37b494-5029-4a32-9ed6-3d5853345f9e None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:55.582490 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c6c8a2-72be-4532-8105-5fea5968d57c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:55.583453 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c6c8a2-72be-4532-8105-5fea5968d57c None admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:55.583840 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c6c8a2-72be-4532-8105-5fea5968d57c None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:55.584188 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 383/766] 10.222.0.64 () {56 vars in 1149 bytes} [Tue Aug 2 09:46:55 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:55.584422 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c6c8a2-72be-4532-8105-5fea5968d57c None admin] PATH_INFO: `/v3/projects/390bd7c4f91c4465b0c7ab0f9e436c4d/users/99655e1f9ca24472a86cd26b0dbd1fc2/roles/1000cf318fa14915b8091599eaead5de` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:55.586686 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c6c8a2-72be-4532-8105-5fea5968d57c None admin] RBAC: Authorizing `identity:create_grant(project_id=390bd7c4f91c4465b0c7ab0f9e436c4d, user_id=99655e1f9ca24472a86cd26b0dbd1fc2, role_id=1000cf318fa14915b8091599eaead5de)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:55.588283 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c6c8a2-72be-4532-8105-5fea5968d57c None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:55.589979 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b37b494-5029-4a32-9ed6-3d5853345f9e None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:55.613333 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 384/767] 10.222.0.64 () {58 vars in 1439 bytes} [Tue Aug 2 09:46:55 2022] PUT /identity/v3/projects/390bd7c4f91c4465b0c7ab0f9e436c4d/users/99655e1f9ca24472a86cd26b0dbd1fc2/roles/1000cf318fa14915b8091599eaead5de => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:46:55.731908 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f84a5bb-eeb1-4f64-86f7-49a3849ad73a None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:55.732450 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f84a5bb-eeb1-4f64-86f7-49a3849ad73a None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:55.732847 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f84a5bb-eeb1-4f64-86f7-49a3849ad73a None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:55.735096 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 385/768] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:46:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:46:55.743111 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b37b494-5029-4a32-9ed6-3d5853345f9e None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:55.743815 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535f0e3e-3615-4d73-8b0b-03622ef8f264 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:55.744204 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535f0e3e-3615-4d73-8b0b-03622ef8f264 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:55.744600 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535f0e3e-3615-4d73-8b0b-03622ef8f264 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:55.761886 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:55.782163 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-535f0e3e-3615-4d73-8b0b-03622ef8f264 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:46:55.783904 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-535f0e3e-3615-4d73-8b0b-03622ef8f264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:46:55.786574 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267b2a98-6b45-4cf5-ab3e-e897c32f3443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:55.787980 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267b2a98-6b45-4cf5-ab3e-e897c32f3443 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:55.788718 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267b2a98-6b45-4cf5-ab3e-e897c32f3443 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:55.789381 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267b2a98-6b45-4cf5-ab3e-e897c32f3443 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:55.908168 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 386/769] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:46:55 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 02 09:46:55.930309 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267b2a98-6b45-4cf5-ab3e-e897c32f3443 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:55.932226 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267b2a98-6b45-4cf5-ab3e-e897c32f3443 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:55.932707 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535f0e3e-3615-4d73-8b0b-03622ef8f264 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:55.946566 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:56.053440 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535f0e3e-3615-4d73-8b0b-03622ef8f264 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:56.061623 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 384/770] 10.222.0.64 () {58 vars in 1282 bytes} [Tue Aug 2 09:46:55 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 02 09:46:56.067936 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267b2a98-6b45-4cf5-ab3e-e897c32f3443 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:56.083407 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:56.089648 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7add5a27-332e-4efc-95c1-5fe6a8917759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:56.090812 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7add5a27-332e-4efc-95c1-5fe6a8917759 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:56.091096 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7add5a27-332e-4efc-95c1-5fe6a8917759 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:56.091370 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7add5a27-332e-4efc-95c1-5fe6a8917759 None admin] PATH_INFO: `/v3/services/network` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:56.092009 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7add5a27-332e-4efc-95c1-5fe6a8917759 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:56.094498 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7add5a27-332e-4efc-95c1-5fe6a8917759 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:56.104856 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-7add5a27-332e-4efc-95c1-5fe6a8917759 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 02 09:46:56.104856 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:56.104856 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 09:46:56.104856 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 09:46:56.104856 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 09:46:56.104856 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 09:46:56.104856 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 09:46:56.104856 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 09:46:56.104856 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 02 09:46:56.104856 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:46:56.104856 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 09:46:56.104856 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:46:56.104856 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:56.104856 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:56.104856 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:56.104856 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:56.104856 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:56.105902 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:56.105902 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:56.105902 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:56.105902 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:56.105902 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:56.105902 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:56.105902 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 09:46:56.105902 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 09:46:56.105902 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 09:46:56.105902 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 09:46:56.105902 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:56.105902 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:56.105902 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:56.105902 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:56.105902 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:56.105902 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:56.107099 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:56.107099 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:56.107099 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:56.107099 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:56.107099 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:56.107099 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:56.107099 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:56.107099 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:56.107099 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:56.107099 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:56.107099 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 09:46:56.107099 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 09:46:56.107099 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 02 09:46:56.107099 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:46:56.107099 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 387/771] 10.222.0.64 () {56 vars in 1111 bytes} [Tue Aug 2 09:46:55 2022] GET /identity/v3/services/network => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 09:46:56.114655 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7add5a27-332e-4efc-95c1-5fe6a8917759 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:56.183484 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267b2a98-6b45-4cf5-ab3e-e897c32f3443 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:56.220787 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdc1b01-e82e-42ef-a177-71fee5c93610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:56.221859 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdc1b01-e82e-42ef-a177-71fee5c93610 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:56.222342 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdc1b01-e82e-42ef-a177-71fee5c93610 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:56.222824 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdc1b01-e82e-42ef-a177-71fee5c93610 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:56.230890 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:56.236542 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdc1b01-e82e-42ef-a177-71fee5c93610 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:56.238287 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdc1b01-e82e-42ef-a177-71fee5c93610 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:56.272803 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-4bdc1b01-e82e-42ef-a177-71fee5c93610 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:46:56.272803 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:56.272803 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:56.272803 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:56.272803 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:56.272803 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:56.272803 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:56.272803 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:56.272803 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:56.272803 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:56.272803 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:56.272803 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:56.272803 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 09:46:56.272803 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 09:46:56.272803 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 09:46:56.273780 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 09:46:56.273780 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:56.273780 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:56.273780 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:56.273780 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:56.273780 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:56.273780 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:56.273780 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:56.273780 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:56.273780 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:56.273780 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:56.273780 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:56.273780 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:56.273780 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:56.273780 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:56.273780 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:56.273780 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:56.274694 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 09:46:56.274694 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 09:46:56.274694 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:46:56.274694 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:46:56.278187 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 385/772] 10.222.0.64 () {56 vars in 1108 bytes} [Tue Aug 2 09:46:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 09:46:56.285361 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdc1b01-e82e-42ef-a177-71fee5c93610 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:56.295025 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7add5a27-332e-4efc-95c1-5fe6a8917759 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:56.377061 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad25a7b-cac3-470d-bf0e-aacf345d8f2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:56.378079 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad25a7b-cac3-470d-bf0e-aacf345d8f2a None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:56.378660 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad25a7b-cac3-470d-bf0e-aacf345d8f2a None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:56.379147 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad25a7b-cac3-470d-bf0e-aacf345d8f2a None admin] PATH_INFO: `/v3/services` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:56.380101 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad25a7b-cac3-470d-bf0e-aacf345d8f2a None admin] RBAC: Adding query filter params (name=network) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:56.380688 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad25a7b-cac3-470d-bf0e-aacf345d8f2a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:56.388278 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad25a7b-cac3-470d-bf0e-aacf345d8f2a None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:56.396415 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:56.398452 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 388/773] 10.222.0.64 () {56 vars in 1112 bytes} [Tue Aug 2 09:46:56 2022] GET /identity/v3/services?name=network => generated 124 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:56.403608 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad25a7b-cac3-470d-bf0e-aacf345d8f2a None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:56.439268 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdc1b01-e82e-42ef-a177-71fee5c93610 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:56.477781 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf585d2-2fbf-4bba-8ad7-3bedec8c8a3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:56.478630 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf585d2-2fbf-4bba-8ad7-3bedec8c8a3e None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:56.478923 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf585d2-2fbf-4bba-8ad7-3bedec8c8a3e None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:56.479200 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf585d2-2fbf-4bba-8ad7-3bedec8c8a3e None admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:56.479900 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf585d2-2fbf-4bba-8ad7-3bedec8c8a3e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:56.480203 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf585d2-2fbf-4bba-8ad7-3bedec8c8a3e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:56.481647 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf585d2-2fbf-4bba-8ad7-3bedec8c8a3e None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:56.495176 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 386/774] 10.222.0.64 () {56 vars in 1109 bytes} [Tue Aug 2 09:46:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:56.500155 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf585d2-2fbf-4bba-8ad7-3bedec8c8a3e None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:56.605558 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf585d2-2fbf-4bba-8ad7-3bedec8c8a3e None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:56.616629 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad25a7b-cac3-470d-bf0e-aacf345d8f2a None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:56.641309 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:56.654191 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43afd72d-5a55-4c0a-aa37-6dd672f70dc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:56.654832 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43afd72d-5a55-4c0a-aa37-6dd672f70dc0 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:56.655114 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43afd72d-5a55-4c0a-aa37-6dd672f70dc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:56.655391 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43afd72d-5a55-4c0a-aa37-6dd672f70dc0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:56.668679 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43afd72d-5a55-4c0a-aa37-6dd672f70dc0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:56.670127 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43afd72d-5a55-4c0a-aa37-6dd672f70dc0 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:56.690711 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eee21a9-107a-42ee-b260-7bd2a9dbe4cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:56.691333 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eee21a9-107a-42ee-b260-7bd2a9dbe4cb None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:56.691615 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eee21a9-107a-42ee-b260-7bd2a9dbe4cb None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:56.691939 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eee21a9-107a-42ee-b260-7bd2a9dbe4cb None admin] PATH_INFO: `/v3/services` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:56.692650 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eee21a9-107a-42ee-b260-7bd2a9dbe4cb None admin] RBAC: Adding query filter params (type=network) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:56.692960 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eee21a9-107a-42ee-b260-7bd2a9dbe4cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:56.694313 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eee21a9-107a-42ee-b260-7bd2a9dbe4cb None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:56.696490 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-43afd72d-5a55-4c0a-aa37-6dd672f70dc0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 09:46:56.696490 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:56.696490 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:56.696490 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:56.696490 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:56.696490 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:56.696490 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:56.696490 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:56.696490 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:56.696490 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:56.696490 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:56.696490 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:56.696490 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 09:46:56.696490 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 09:46:56.696490 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 09:46:56.697390 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 09:46:56.697390 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:56.697390 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:56.697390 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:56.697390 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:56.697390 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:56.697390 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:56.697390 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:56.697390 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:56.697390 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:56.697390 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:56.697390 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:56.697390 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:56.697390 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:56.697390 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:56.697390 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:56.697390 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:56.698301 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 09:46:56.698301 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 09:46:56.698301 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 09:46:56.698301 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 09:46:56.698301 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 09:46:56.698301 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 09:46:56.698301 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 09:46:56.698301 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:46:56.699834 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 387/775] 10.222.0.64 () {56 vars in 1111 bytes} [Tue Aug 2 09:46:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 09:46:56.701831 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 389/776] 10.222.0.64 () {56 vars in 1112 bytes} [Tue Aug 2 09:46:56 2022] GET /identity/v3/services?type=network => generated 350 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:46:56.704175 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43afd72d-5a55-4c0a-aa37-6dd672f70dc0 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:56.707885 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eee21a9-107a-42ee-b260-7bd2a9dbe4cb None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:56.712229 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:56.830692 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43afd72d-5a55-4c0a-aa37-6dd672f70dc0 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:56.866731 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3d05c6-a446-469e-9009-10feddaab162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:56.867359 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3d05c6-a446-469e-9009-10feddaab162 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:56.867644 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3d05c6-a446-469e-9009-10feddaab162 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:56.867919 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3d05c6-a446-469e-9009-10feddaab162 None admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:56.868748 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3d05c6-a446-469e-9009-10feddaab162 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:56.869061 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3d05c6-a446-469e-9009-10feddaab162 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:56.870516 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3d05c6-a446-469e-9009-10feddaab162 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:56.886343 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 388/777] 10.222.0.64 () {56 vars in 1148 bytes} [Tue Aug 2 09:46:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:46:56.890061 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3d05c6-a446-469e-9009-10feddaab162 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:56.902750 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eee21a9-107a-42ee-b260-7bd2a9dbe4cb None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:56.936816 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:56.942916 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be170bbc-08a2-4009-bea4-3f2f1327287e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:56.943937 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be170bbc-08a2-4009-bea4-3f2f1327287e None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:56.944533 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be170bbc-08a2-4009-bea4-3f2f1327287e None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:56.945012 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be170bbc-08a2-4009-bea4-3f2f1327287e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:56.945974 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be170bbc-08a2-4009-bea4-3f2f1327287e None admin] RBAC: Adding query filter params (interface=public, service_id=624af1066f2443d3abfd7c7be1359564, region_id=RegionOne) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:56.946491 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be170bbc-08a2-4009-bea4-3f2f1327287e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:56.948233 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be170bbc-08a2-4009-bea4-3f2f1327287e None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:56.957362 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 390/778] 10.222.0.64 () {56 vars in 1251 bytes} [Tue Aug 2 09:46:56 2022] GET /identity/v3/endpoints?service_id=624af1066f2443d3abfd7c7be1359564&interface=public®ion_id=RegionOne => generated 194 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:56.969217 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be170bbc-08a2-4009-bea4-3f2f1327287e None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:56.993309 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3d05c6-a446-469e-9009-10feddaab162 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:57.034956 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934258f6-375c-4bf5-9b76-b3c928788349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:57.036208 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934258f6-375c-4bf5-9b76-b3c928788349 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:57.036886 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934258f6-375c-4bf5-9b76-b3c928788349 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:57.037529 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934258f6-375c-4bf5-9b76-b3c928788349 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:57.039205 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934258f6-375c-4bf5-9b76-b3c928788349 None admin] RBAC: Adding query filter params (role.id=5704af09562a457fa35364e8af9481f1, scope.project.id=390bd7c4f91c4465b0c7ab0f9e436c4d, user.id=5338bc8c594e4bc9bc41019226bf8d87) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:57.039882 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934258f6-375c-4bf5-9b76-b3c928788349 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:57.041904 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934258f6-375c-4bf5-9b76-b3c928788349 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:57.052497 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 389/779] 10.222.0.64 () {56 vars in 1374 bytes} [Tue Aug 2 09:46:56 2022] GET /identity/v3/role_assignments?user.id=5338bc8c594e4bc9bc41019226bf8d87&scope.project.id=390bd7c4f91c4465b0c7ab0f9e436c4d&role.id=5704af09562a457fa35364e8af9481f1 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:57.108834 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:57.172367 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be170bbc-08a2-4009-bea4-3f2f1327287e None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:57.249746 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68524183-8556-44bc-88f4-945cbb9caeb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:57.251678 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68524183-8556-44bc-88f4-945cbb9caeb3 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:57.252148 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68524183-8556-44bc-88f4-945cbb9caeb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:57.252643 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68524183-8556-44bc-88f4-945cbb9caeb3 None admin] PATH_INFO: `/v3/services` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:57.253469 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68524183-8556-44bc-88f4-945cbb9caeb3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:57.255024 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68524183-8556-44bc-88f4-945cbb9caeb3 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:57.261305 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 391/780] 10.222.0.64 () {56 vars in 1087 bytes} [Tue Aug 2 09:46:56 2022] GET /identity/v3/services => generated 772 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:57.486424 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9e4e55-0ea9-45b8-bcd1-c8e1dcd9db87 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:57.487287 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9e4e55-0ea9-45b8-bcd1-c8e1dcd9db87 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:57.487717 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9e4e55-0ea9-45b8-bcd1-c8e1dcd9db87 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:57.489641 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 390/781] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:46:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:46:57.496053 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9441239-92e4-46ad-b327-d71f3021a6aa None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:57.496971 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9441239-92e4-46ad-b327-d71f3021a6aa None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:57.497400 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9441239-92e4-46ad-b327-d71f3021a6aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:57.524717 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-c9441239-92e4-46ad-b327-d71f3021a6aa None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:46:57.525982 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-c9441239-92e4-46ad-b327-d71f3021a6aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:46:57.613843 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 392/782] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:46:57 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 02 09:46:57.636513 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9e4e55-0ea9-45b8-bcd1-c8e1dcd9db87 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:57.778154 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9e4e55-0ea9-45b8-bcd1-c8e1dcd9db87 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:57.788835 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:57.831281 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c4e55c-1e43-46c5-ae60-19da022c3404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:57.832325 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c4e55c-1e43-46c5-ae60-19da022c3404 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:57.832888 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c4e55c-1e43-46c5-ae60-19da022c3404 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:57.833364 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c4e55c-1e43-46c5-ae60-19da022c3404 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:57.852614 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c4e55c-1e43-46c5-ae60-19da022c3404 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 09:46:57.857504 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-29c4e55c-1e43-46c5-ae60-19da022c3404 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 09:46:57.857504 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:57.857504 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:57.857504 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:57.857504 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:57.857504 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:57.857504 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:57.857504 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:57.857504 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:57.857504 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:57.857504 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:57.857504 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:57.857504 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 09:46:57.857504 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 09:46:57.857504 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 09:46:57.857504 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 09:46:57.858503 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 09:46:57.858503 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 09:46:57.858503 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 09:46:57.858503 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 09:46:57.858503 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:57.858503 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:57.858503 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:57.858503 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:57.858503 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:57.858503 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:57.858503 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:57.858503 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:57.858503 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:57.858503 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:57.858503 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:57.858503 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:57.859934 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:57.859934 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:57.859934 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:57.859934 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:57.859934 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 09:46:57.859934 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 09:46:57.859934 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 09:46:57.859934 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 09:46:57.859934 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 09:46:57.859934 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 09:46:57.859934 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 09:46:57.859934 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:46:57.859934 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 391/783] 10.222.0.64 () {56 vars in 1096 bytes} [Tue Aug 2 09:46:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:46:57.865356 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9441239-92e4-46ad-b327-d71f3021a6aa None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:57.971654 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:58.025201 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9441239-92e4-46ad-b327-d71f3021a6aa None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:58.073561 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda4433b-74e9-48a4-bdc4-06b03a2452da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:58.074187 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda4433b-74e9-48a4-bdc4-06b03a2452da None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:58.074468 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda4433b-74e9-48a4-bdc4-06b03a2452da None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:58.074740 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda4433b-74e9-48a4-bdc4-06b03a2452da None admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:58.075479 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda4433b-74e9-48a4-bdc4-06b03a2452da None admin] RBAC: Adding query filter params (name=admin) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:58.075783 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda4433b-74e9-48a4-bdc4-06b03a2452da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:58.077184 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda4433b-74e9-48a4-bdc4-06b03a2452da None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:58.091560 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 393/784] 10.222.0.64 () {56 vars in 1099 bytes} [Tue Aug 2 09:46:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:58.097295 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c4e55c-1e43-46c5-ae60-19da022c3404 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:58.120696 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4ac25f-99e1-469c-bb63-23cb06eeeaa5 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:58.120696 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4ac25f-99e1-469c-bb63-23cb06eeeaa5 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:58.121101 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4ac25f-99e1-469c-bb63-23cb06eeeaa5 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:58.122406 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 394/785] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:46:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:46:58.127608 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aace90-3394-4664-81f1-c08c6cf4858d None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:58.127911 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aace90-3394-4664-81f1-c08c6cf4858d None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:58.128177 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aace90-3394-4664-81f1-c08c6cf4858d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:58.161873 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-86aace90-3394-4664-81f1-c08c6cf4858d None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:46:58.164472 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-86aace90-3394-4664-81f1-c08c6cf4858d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:46:58.235920 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c4e55c-1e43-46c5-ae60-19da022c3404 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:58.271148 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef301f38-88e3-4ade-8aa7-dd1a3ddbb61b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:58.271786 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef301f38-88e3-4ade-8aa7-dd1a3ddbb61b None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:58.272056 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef301f38-88e3-4ade-8aa7-dd1a3ddbb61b None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:58.272363 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef301f38-88e3-4ade-8aa7-dd1a3ddbb61b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:58.286557 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef301f38-88e3-4ade-8aa7-dd1a3ddbb61b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:58.288014 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef301f38-88e3-4ade-8aa7-dd1a3ddbb61b None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:58.306522 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-ef301f38-88e3-4ade-8aa7-dd1a3ddbb61b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:46:58.306522 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:58.306522 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:58.306522 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:58.306522 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:58.306522 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:58.306522 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:58.306522 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:58.306522 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:58.306522 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:58.306522 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:58.306522 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:58.306522 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 09:46:58.306522 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 09:46:58.306522 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 09:46:58.307520 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 09:46:58.307520 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:58.307520 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:58.307520 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:58.307520 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:58.307520 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:58.307520 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:58.307520 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:58.307520 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:58.307520 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:58.307520 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:58.307520 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:58.307520 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:58.307520 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:58.307520 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:58.307520 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:58.307520 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:58.308493 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 09:46:58.308493 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 09:46:58.308493 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:46:58.308493 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:46:58.308493 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 392/786] 10.222.0.64 () {56 vars in 1108 bytes} [Tue Aug 2 09:46:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:46:58.309477 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2cf284-0277-4b4d-81c9-e1fdd2a23c89 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:58.309477 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2cf284-0277-4b4d-81c9-e1fdd2a23c89 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:58.309663 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2cf284-0277-4b4d-81c9-e1fdd2a23c89 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:58.311686 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 393/787] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:46:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:46:58.314797 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2cf284-0277-4b4d-81c9-e1fdd2a23c89 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:58.323721 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 395/788] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:46:58 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 09:46:58.325374 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc9a1cb-5aa9-4f59-9796-44a52eba9274 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:58.325902 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc9a1cb-5aa9-4f59-9796-44a52eba9274 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:58.326389 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc9a1cb-5aa9-4f59-9796-44a52eba9274 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:58.342252 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:58.354970 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-0bc9a1cb-5aa9-4f59-9796-44a52eba9274 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:46:58.357692 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-0bc9a1cb-5aa9-4f59-9796-44a52eba9274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:46:58.434771 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:58.446557 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2cf284-0277-4b4d-81c9-e1fdd2a23c89 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:58.471428 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 396/789] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:46:58 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 09:46:58.472837 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc9a1cb-5aa9-4f59-9796-44a52eba9274 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:58.506725 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3887b0bc-82cb-4928-9696-242c959baac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:58.507414 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3887b0bc-82cb-4928-9696-242c959baac9 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:58.507704 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3887b0bc-82cb-4928-9696-242c959baac9 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:58.507986 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3887b0bc-82cb-4928-9696-242c959baac9 None admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:58.508748 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3887b0bc-82cb-4928-9696-242c959baac9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:58.509059 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3887b0bc-82cb-4928-9696-242c959baac9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:58.510501 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3887b0bc-82cb-4928-9696-242c959baac9 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:58.528164 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 394/790] 10.222.0.64 () {56 vars in 1109 bytes} [Tue Aug 2 09:46:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:46:58.529227 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3887b0bc-82cb-4928-9696-242c959baac9 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:58.603259 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc9a1cb-5aa9-4f59-9796-44a52eba9274 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:58.638231 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:58.644613 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420898a1-d65d-4d73-8fa2-641a094478c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:58.645210 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420898a1-d65d-4d73-8fa2-641a094478c7 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:58.645583 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420898a1-d65d-4d73-8fa2-641a094478c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:58.645932 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420898a1-d65d-4d73-8fa2-641a094478c7 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:58.646717 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420898a1-d65d-4d73-8fa2-641a094478c7 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:58.648219 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420898a1-d65d-4d73-8fa2-641a094478c7 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:58.658333 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-420898a1-d65d-4d73-8fa2-641a094478c7 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 02 09:46:58.658333 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:58.658333 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 09:46:58.658333 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 09:46:58.658333 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 09:46:58.658333 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 09:46:58.658333 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 09:46:58.658333 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 09:46:58.658333 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 02 09:46:58.658333 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:46:58.658333 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 09:46:58.658333 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:46:58.658333 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:58.658333 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:58.658333 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:58.658333 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:58.659337 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:58.659337 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:58.659337 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:58.659337 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:58.659337 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:58.659337 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:58.659337 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:58.659337 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 09:46:58.659337 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 09:46:58.659337 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 09:46:58.659337 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 09:46:58.659337 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:58.659337 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:58.659337 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:58.659337 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:58.659337 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:58.660209 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:58.660209 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:58.660209 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:58.660209 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:58.660209 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:58.660209 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:58.660209 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:58.660209 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:58.660209 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:58.660209 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:58.660209 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:58.660209 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 09:46:58.660209 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 09:46:58.660209 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 02 09:46:58.660209 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:46:58.660209 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 397/791] 10.222.0.64 () {56 vars in 1129 bytes} [Tue Aug 2 09:46:58 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:46:58.661116 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420898a1-d65d-4d73-8fa2-641a094478c7 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:58.667180 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3887b0bc-82cb-4928-9696-242c959baac9 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:58.707102 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26ae82f-b4e1-415c-8a9f-9dee5be1039a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:58.708004 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26ae82f-b4e1-415c-8a9f-9dee5be1039a None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:58.708557 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26ae82f-b4e1-415c-8a9f-9dee5be1039a None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:58.708940 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26ae82f-b4e1-415c-8a9f-9dee5be1039a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:58.721072 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26ae82f-b4e1-415c-8a9f-9dee5be1039a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 09:46:58.726108 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:58.727182 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-b26ae82f-b4e1-415c-8a9f-9dee5be1039a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 09:46:58.727182 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:58.727182 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:58.727182 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:58.727182 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:58.727182 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:58.727182 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:58.727182 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:58.727182 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:58.727182 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:58.727182 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:58.727182 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:58.727182 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 09:46:58.727182 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 09:46:58.727182 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 09:46:58.727182 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 09:46:58.728849 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 09:46:58.728849 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 09:46:58.728849 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 09:46:58.728849 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 09:46:58.728849 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:58.728849 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:58.728849 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:58.728849 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:58.728849 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:58.728849 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:58.728849 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:58.728849 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:58.728849 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:58.728849 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:58.728849 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:58.728849 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:58.729934 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:58.729934 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:58.729934 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:58.729934 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:58.729934 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 09:46:58.729934 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 09:46:58.729934 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 09:46:58.729934 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 09:46:58.729934 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 09:46:58.729934 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 09:46:58.729934 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 09:46:58.729934 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:46:58.739658 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 395/792] 10.222.0.64 () {56 vars in 1096 bytes} [Tue Aug 2 09:46:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:46:58.741054 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26ae82f-b4e1-415c-8a9f-9dee5be1039a None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:58.806546 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420898a1-d65d-4d73-8fa2-641a094478c7 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:58.855000 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c7e653-d8b6-4622-bb0b-c25784ba02e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:58.855940 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c7e653-d8b6-4622-bb0b-c25784ba02e0 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:58.856426 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c7e653-d8b6-4622-bb0b-c25784ba02e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:58.856873 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c7e653-d8b6-4622-bb0b-c25784ba02e0 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:58.869098 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c7e653-d8b6-4622-bb0b-c25784ba02e0 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:58.870923 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c7e653-d8b6-4622-bb0b-c25784ba02e0 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:58.871408 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26ae82f-b4e1-415c-8a9f-9dee5be1039a None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:58.886709 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-f6c7e653-d8b6-4622-bb0b-c25784ba02e0 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 02 09:46:58.886709 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:58.886709 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:58.886709 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:58.886709 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:58.886709 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:58.886709 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:58.886709 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:58.886709 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:58.886709 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:58.886709 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:58.886709 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:58.886709 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 09:46:58.886709 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 09:46:58.886709 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 09:46:58.886709 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 09:46:58.887946 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:58.887946 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:58.887946 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 09:46:58.887946 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 09:46:58.887946 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 09:46:58.887946 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 09:46:58.887946 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:58.887946 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:58.887946 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:58.887946 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:58.887946 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:58.887946 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:58.887946 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:58.887946 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:58.887946 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:58.887946 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:58.887946 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:58.887946 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:58.889196 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:58.889196 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:58.889196 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 09:46:58.889196 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 09:46:58.889196 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 09:46:58.889196 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 09:46:58.889196 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 09:46:58.889196 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 09:46:58.889196 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 02 09:46:58.889196 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:46:58.893175 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 398/793] 10.222.0.64 () {56 vars in 1096 bytes} [Tue Aug 2 09:46:58 2022] GET /identity/v3/users/swift => generated 83 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:46:58.894235 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c7e653-d8b6-4622-bb0b-c25784ba02e0 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:58.931386 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be841dd-81b9-4ee7-93d3-2123707b2533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:58.932012 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be841dd-81b9-4ee7-93d3-2123707b2533 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:58.932319 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be841dd-81b9-4ee7-93d3-2123707b2533 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:58.932603 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be841dd-81b9-4ee7-93d3-2123707b2533 None admin] PATH_INFO: `/v3/services` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:58.933299 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be841dd-81b9-4ee7-93d3-2123707b2533 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:58.933606 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be841dd-81b9-4ee7-93d3-2123707b2533 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:58.934990 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be841dd-81b9-4ee7-93d3-2123707b2533 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:58.941172 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 396/794] 10.222.0.64 () {56 vars in 1124 bytes} [Tue Aug 2 09:46:58 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:58.942183 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be841dd-81b9-4ee7-93d3-2123707b2533 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:58.946635 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:59.018154 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c7e653-d8b6-4622-bb0b-c25784ba02e0 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:59.026198 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:59.056060 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be841dd-81b9-4ee7-93d3-2123707b2533 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:59.056764 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a2bef4-1d46-4977-b17b-1f8be74aeb5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:59.057626 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a2bef4-1d46-4977-b17b-1f8be74aeb5d None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:59.058123 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a2bef4-1d46-4977-b17b-1f8be74aeb5d None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:59.058637 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a2bef4-1d46-4977-b17b-1f8be74aeb5d None admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:59.059579 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a2bef4-1d46-4977-b17b-1f8be74aeb5d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:59.060100 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a2bef4-1d46-4977-b17b-1f8be74aeb5d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:59.061772 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a2bef4-1d46-4977-b17b-1f8be74aeb5d None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:59.077260 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 399/795] 10.222.0.64 () {56 vars in 1099 bytes} [Tue Aug 2 09:46:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:59.079660 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a2bef4-1d46-4977-b17b-1f8be74aeb5d None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:59.104385 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bec82f-7ebb-4dd2-8f04-79c74900633d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:59.105107 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bec82f-7ebb-4dd2-8f04-79c74900633d None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:59.105339 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bec82f-7ebb-4dd2-8f04-79c74900633d None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:59.105617 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bec82f-7ebb-4dd2-8f04-79c74900633d None admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:59.106438 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bec82f-7ebb-4dd2-8f04-79c74900633d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:59.106752 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bec82f-7ebb-4dd2-8f04-79c74900633d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:59.108195 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bec82f-7ebb-4dd2-8f04-79c74900633d None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:59.128520 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 397/796] 10.222.0.64 () {56 vars in 1135 bytes} [Tue Aug 2 09:46:58 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:59.129380 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bec82f-7ebb-4dd2-8f04-79c74900633d None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:59.194111 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a2bef4-1d46-4977-b17b-1f8be74aeb5d None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:59.202727 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:59.232607 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda0cf46-c8a3-4774-a4f3-c0ece41b130e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:59.233613 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda0cf46-c8a3-4774-a4f3-c0ece41b130e None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:59.234181 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda0cf46-c8a3-4774-a4f3-c0ece41b130e None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:59.235193 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda0cf46-c8a3-4774-a4f3-c0ece41b130e None admin] PATH_INFO: `/v3/services` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:59.236121 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda0cf46-c8a3-4774-a4f3-c0ece41b130e None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:59.236668 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda0cf46-c8a3-4774-a4f3-c0ece41b130e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:59.238282 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda0cf46-c8a3-4774-a4f3-c0ece41b130e None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:59.244205 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 400/797] 10.222.0.64 () {56 vars in 1124 bytes} [Tue Aug 2 09:46:59 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:59.245392 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda0cf46-c8a3-4774-a4f3-c0ece41b130e None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:59.247020 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bec82f-7ebb-4dd2-8f04-79c74900633d None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:59.280792 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:59.288478 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ec7ac2-a8fc-4cc8-956d-b6405db10b6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:59.289219 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ec7ac2-a8fc-4cc8-956d-b6405db10b6d None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:59.289512 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ec7ac2-a8fc-4cc8-956d-b6405db10b6d None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:59.289796 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ec7ac2-a8fc-4cc8-956d-b6405db10b6d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:59.302411 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ec7ac2-a8fc-4cc8-956d-b6405db10b6d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:59.303950 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ec7ac2-a8fc-4cc8-956d-b6405db10b6d None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:59.321217 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-57ec7ac2-a8fc-4cc8-956d-b6405db10b6d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:46:59.321217 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:59.321217 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:59.321217 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:59.321217 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:59.321217 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:59.321217 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:59.321217 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:59.321217 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:59.321217 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:59.321217 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:59.321217 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:59.321217 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 09:46:59.321217 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 09:46:59.321217 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 09:46:59.323087 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 09:46:59.323087 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:59.323087 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:59.323087 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:59.323087 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:59.323087 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:59.323087 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:59.323087 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:59.323087 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:59.323087 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:59.323087 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:59.323087 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:59.323087 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:59.323087 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:59.323087 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:59.323087 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:59.323087 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:59.324006 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 09:46:59.324006 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 09:46:59.324006 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:46:59.324006 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:46:59.324006 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 398/798] 10.222.0.64 () {56 vars in 1108 bytes} [Tue Aug 2 09:46:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:46:59.324006 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ec7ac2-a8fc-4cc8-956d-b6405db10b6d None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:59.365782 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda0cf46-c8a3-4774-a4f3-c0ece41b130e None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:59.405113 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25309ccc-d3b5-4a3a-939a-9de4842af9ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:59.405771 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25309ccc-d3b5-4a3a-939a-9de4842af9ee None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:59.406610 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25309ccc-d3b5-4a3a-939a-9de4842af9ee None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:59.406916 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25309ccc-d3b5-4a3a-939a-9de4842af9ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:59.418434 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25309ccc-d3b5-4a3a-939a-9de4842af9ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:59.419899 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25309ccc-d3b5-4a3a-939a-9de4842af9ee None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:59.434748 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-25309ccc-d3b5-4a3a-939a-9de4842af9ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:46:59.434748 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:59.434748 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:59.434748 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:59.434748 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:59.434748 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:59.434748 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:59.434748 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:59.434748 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:59.434748 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:59.434748 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:59.434748 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:59.434748 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 09:46:59.434748 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 09:46:59.434748 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 09:46:59.435695 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 09:46:59.435695 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:59.435695 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:59.435695 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:59.435695 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:59.435695 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:59.435695 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:59.435695 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:59.435695 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:59.435695 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:59.435695 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:59.435695 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:59.435695 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:59.435695 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:59.435695 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:59.435695 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:59.435695 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:59.436690 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 09:46:59.436690 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 09:46:59.436690 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:46:59.436690 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:46:59.436690 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 401/799] 10.222.0.64 () {56 vars in 1108 bytes} [Tue Aug 2 09:46:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:46:59.438041 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135fc83f-9822-437e-abc9-b57e6a8c33a5 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:59.438328 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135fc83f-9822-437e-abc9-b57e6a8c33a5 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:59.438601 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135fc83f-9822-437e-abc9-b57e6a8c33a5 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:59.439961 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 402/800] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:46:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:46:59.444524 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135fc83f-9822-437e-abc9-b57e6a8c33a5 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:59.449227 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ec7ac2-a8fc-4cc8-956d-b6405db10b6d None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:59.463575 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:59.488019 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70de7359-2e15-4a74-b85d-a4f6b4b8c773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:59.489167 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70de7359-2e15-4a74-b85d-a4f6b4b8c773 None admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:59.489456 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70de7359-2e15-4a74-b85d-a4f6b4b8c773 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:59.489733 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70de7359-2e15-4a74-b85d-a4f6b4b8c773 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:59.490446 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70de7359-2e15-4a74-b85d-a4f6b4b8c773 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:59.492283 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70de7359-2e15-4a74-b85d-a4f6b4b8c773 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:59.524585 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 399/801] 10.222.0.64 () {60 vars in 1144 bytes} [Tue Aug 2 09:46:59 2022] POST /identity/v3/endpoints => generated 346 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:46:59.525586 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70de7359-2e15-4a74-b85d-a4f6b4b8c773 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:59.548726 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:59.602412 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135fc83f-9822-437e-abc9-b57e6a8c33a5 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:59.645757 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3d6883-d3b8-4537-809a-df15f94d6fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:59.646386 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3d6883-d3b8-4537-809a-df15f94d6fc2 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:59.646646 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3d6883-d3b8-4537-809a-df15f94d6fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:59.646929 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3d6883-d3b8-4537-809a-df15f94d6fc2 None admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:59.647651 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3d6883-d3b8-4537-809a-df15f94d6fc2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:59.647962 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3d6883-d3b8-4537-809a-df15f94d6fc2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:59.649522 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3d6883-d3b8-4537-809a-df15f94d6fc2 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:59.667769 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70de7359-2e15-4a74-b85d-a4f6b4b8c773 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:59.669515 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 403/802] 10.222.0.64 () {56 vars in 1109 bytes} [Tue Aug 2 09:46:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:59.672476 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd2138c-4a7b-4d6f-aaa8-5380bf73d048 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:59.672476 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd2138c-4a7b-4d6f-aaa8-5380bf73d048 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:59.672476 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd2138c-4a7b-4d6f-aaa8-5380bf73d048 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:59.696267 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-4dd2138c-4a7b-4d6f-aaa8-5380bf73d048 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:46:59.698049 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-4dd2138c-4a7b-4d6f-aaa8-5380bf73d048 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:46:59.713857 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6845a98e-a9f9-412b-a13e-2f303f784281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:59.713857 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6845a98e-a9f9-412b-a13e-2f303f784281 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:59.713857 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6845a98e-a9f9-412b-a13e-2f303f784281 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:59.713857 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6845a98e-a9f9-412b-a13e-2f303f784281 None admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:59.714570 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6845a98e-a9f9-412b-a13e-2f303f784281 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:46:59.714885 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6845a98e-a9f9-412b-a13e-2f303f784281 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:59.716334 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6845a98e-a9f9-412b-a13e-2f303f784281 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:59.730646 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 400/803] 10.222.0.64 () {56 vars in 1109 bytes} [Tue Aug 2 09:46:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:46:59.731602 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6845a98e-a9f9-412b-a13e-2f303f784281 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:59.762904 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:59.777569 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 404/804] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:46:59 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 09:46:59.779242 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07221164-5b5a-43a5-a78e-53933f716e29 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:59.779537 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07221164-5b5a-43a5-a78e-53933f716e29 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:59.780141 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07221164-5b5a-43a5-a78e-53933f716e29 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:59.782291 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 405/805] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:46:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:46:59.782989 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07221164-5b5a-43a5-a78e-53933f716e29 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:46:59.856437 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6845a98e-a9f9-412b-a13e-2f303f784281 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:59.860453 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:46:59.892131 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2053be-6e81-420e-b3b0-47719524c81a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:59.892855 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2053be-6e81-420e-b3b0-47719524c81a None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:59.893203 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2053be-6e81-420e-b3b0-47719524c81a None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:59.893549 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2053be-6e81-420e-b3b0-47719524c81a None admin] PATH_INFO: `/v3/users/nova` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:59.900913 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2053be-6e81-420e-b3b0-47719524c81a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:59.901207 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07221164-5b5a-43a5-a78e-53933f716e29 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:46:59.902443 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2053be-6e81-420e-b3b0-47719524c81a None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:59.917488 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-8e2053be-6e81-420e-b3b0-47719524c81a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 02 09:46:59.917488 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:59.917488 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:59.917488 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:59.917488 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:59.917488 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:59.917488 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:59.917488 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:59.917488 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:59.917488 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:59.917488 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:59.917488 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:59.917488 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 09:46:59.917488 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 09:46:59.917488 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 09:46:59.917488 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 09:46:59.918648 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:59.918648 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:59.918648 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 09:46:59.918648 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 09:46:59.918648 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 09:46:59.918648 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 09:46:59.918648 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:59.918648 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:59.918648 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:59.918648 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:59.918648 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:59.918648 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:59.918648 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:59.918648 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:59.918648 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:59.918648 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:59.918648 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:59.918648 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:59.919829 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:59.919829 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:59.919829 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 09:46:59.919829 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 09:46:59.919829 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 09:46:59.919829 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 09:46:59.919829 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 09:46:59.919829 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 09:46:59.919829 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 02 09:46:59.919829 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:46:59.922356 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 401/806] 10.222.0.64 () {56 vars in 1093 bytes} [Tue Aug 2 09:46:59 2022] GET /identity/v3/users/nova => generated 82 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:46:59.923669 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c3da48-874a-45de-97f4-3fefebc4a25c None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:59.923953 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c3da48-874a-45de-97f4-3fefebc4a25c None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:59.924228 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c3da48-874a-45de-97f4-3fefebc4a25c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:59.941981 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-a8c3da48-874a-45de-97f4-3fefebc4a25c None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:46:59.942173 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acde480e-090b-4491-afc6-2eed875a7bdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:46:59.942802 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acde480e-090b-4491-afc6-2eed875a7bdd None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:46:59.943090 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acde480e-090b-4491-afc6-2eed875a7bdd None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:46:59.943189 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-a8c3da48-874a-45de-97f4-3fefebc4a25c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:46:59.943367 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acde480e-090b-4491-afc6-2eed875a7bdd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:46:59.950819 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acde480e-090b-4491-afc6-2eed875a7bdd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:46:59.952275 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acde480e-090b-4491-afc6-2eed875a7bdd None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:46:59.963648 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-acde480e-090b-4491-afc6-2eed875a7bdd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 09:46:59.963648 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:46:59.963648 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:46:59.963648 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:46:59.963648 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:46:59.963648 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:46:59.963648 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:46:59.963648 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:46:59.963648 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:46:59.963648 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:46:59.963648 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:46:59.963648 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:46:59.963648 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 09:46:59.963648 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 09:46:59.963648 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 09:46:59.964720 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 09:46:59.964720 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:46:59.964720 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:46:59.964720 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:46:59.964720 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:46:59.964720 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:46:59.964720 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:46:59.964720 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:46:59.964720 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:46:59.964720 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:46:59.964720 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:46:59.964720 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:46:59.964720 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:46:59.964720 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:46:59.964720 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:46:59.964720 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:46:59.964720 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:46:59.965620 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 09:46:59.965620 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 09:46:59.965620 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 09:46:59.965620 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 09:46:59.965620 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 09:46:59.965620 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 09:46:59.965620 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 09:46:59.965620 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:46:59.965620 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 406/807] 10.222.0.64 () {56 vars in 1111 bytes} [Tue Aug 2 09:46:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:46:59.965620 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acde480e-090b-4491-afc6-2eed875a7bdd None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:00.020277 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 402/808] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:46:59 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 09:47:00.021378 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c3da48-874a-45de-97f4-3fefebc4a25c None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:00.046364 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:00.077529 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acde480e-090b-4491-afc6-2eed875a7bdd None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:00.107393 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:00.117502 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df5df93-9a3f-45aa-ad7b-92466c4adae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:00.118138 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df5df93-9a3f-45aa-ad7b-92466c4adae2 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:00.118424 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df5df93-9a3f-45aa-ad7b-92466c4adae2 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:00.118708 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df5df93-9a3f-45aa-ad7b-92466c4adae2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:00.130001 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c3da48-874a-45de-97f4-3fefebc4a25c None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:00.130144 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df5df93-9a3f-45aa-ad7b-92466c4adae2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 09:47:00.135351 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-8df5df93-9a3f-45aa-ad7b-92466c4adae2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 09:47:00.135351 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:47:00.135351 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:47:00.135351 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:47:00.135351 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:47:00.135351 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:47:00.135351 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:47:00.135351 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:47:00.135351 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:47:00.135351 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:47:00.135351 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:47:00.135351 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:47:00.135351 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 09:47:00.135351 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 09:47:00.135351 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 09:47:00.135351 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 09:47:00.136581 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 09:47:00.136581 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 09:47:00.136581 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 09:47:00.136581 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 09:47:00.136581 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:47:00.136581 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:47:00.136581 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:47:00.136581 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:47:00.136581 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:47:00.136581 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:47:00.136581 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:47:00.136581 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:47:00.136581 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:47:00.136581 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:47:00.136581 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:47:00.136581 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:47:00.137479 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:47:00.137479 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:47:00.137479 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:47:00.137479 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:47:00.137479 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 09:47:00.137479 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 09:47:00.137479 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 09:47:00.137479 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 09:47:00.137479 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 09:47:00.137479 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 09:47:00.137479 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 09:47:00.137479 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:47:00.137479 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 407/809] 10.222.0.64 () {56 vars in 1102 bytes} [Tue Aug 2 09:46:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:47:00.138183 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df5df93-9a3f-45aa-ad7b-92466c4adae2 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:00.170273 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c36d760-1e89-490d-a73d-24c4905ee86a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:00.170812 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c36d760-1e89-490d-a73d-24c4905ee86a None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:00.171093 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c36d760-1e89-490d-a73d-24c4905ee86a None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:00.171365 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c36d760-1e89-490d-a73d-24c4905ee86a None admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:00.172801 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c36d760-1e89-490d-a73d-24c4905ee86a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:47:00.173117 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c36d760-1e89-490d-a73d-24c4905ee86a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:00.174535 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c36d760-1e89-490d-a73d-24c4905ee86a None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:00.192637 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 403/810] 10.222.0.64 () {56 vars in 1133 bytes} [Tue Aug 2 09:47:00 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:47:00.192637 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c36d760-1e89-490d-a73d-24c4905ee86a None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:00.242739 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df5df93-9a3f-45aa-ad7b-92466c4adae2 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:00.277460 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5085e87f-a8aa-44bf-a157-655a4f822901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:00.278088 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5085e87f-a8aa-44bf-a157-655a4f822901 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:00.278380 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5085e87f-a8aa-44bf-a157-655a4f822901 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:00.278668 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5085e87f-a8aa-44bf-a157-655a4f822901 None admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:00.279424 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5085e87f-a8aa-44bf-a157-655a4f822901 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:47:00.279737 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5085e87f-a8aa-44bf-a157-655a4f822901 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:00.280706 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:00.282668 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5085e87f-a8aa-44bf-a157-655a4f822901 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:00.294971 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 408/811] 10.222.0.64 () {56 vars in 1148 bytes} [Tue Aug 2 09:47:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:47:00.296362 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5085e87f-a8aa-44bf-a157-655a4f822901 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:00.319665 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c36d760-1e89-490d-a73d-24c4905ee86a None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:00.361260 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:00.361986 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d12920-f310-4d13-82d8-b089254c8d63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:00.362830 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d12920-f310-4d13-82d8-b089254c8d63 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:00.363268 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d12920-f310-4d13-82d8-b089254c8d63 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:00.363698 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d12920-f310-4d13-82d8-b089254c8d63 None admin] PATH_INFO: `/v3/services/network` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:00.364543 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d12920-f310-4d13-82d8-b089254c8d63 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:00.366020 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d12920-f310-4d13-82d8-b089254c8d63 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:00.376154 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-27d12920-f310-4d13-82d8-b089254c8d63 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 02 09:47:00.376154 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:47:00.376154 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 09:47:00.376154 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 09:47:00.376154 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 09:47:00.376154 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 09:47:00.376154 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 09:47:00.376154 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 09:47:00.376154 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 02 09:47:00.376154 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:47:00.376154 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 09:47:00.376154 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:47:00.376154 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:47:00.376154 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:47:00.376154 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:47:00.376154 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:47:00.376154 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:47:00.377270 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:47:00.377270 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:47:00.377270 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:47:00.377270 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:47:00.377270 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:47:00.377270 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:47:00.377270 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 09:47:00.377270 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 09:47:00.377270 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 09:47:00.377270 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 09:47:00.377270 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:47:00.377270 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:47:00.377270 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:47:00.377270 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:47:00.377270 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:47:00.377270 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:47:00.378121 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:47:00.378121 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:47:00.378121 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:47:00.378121 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:47:00.378121 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:47:00.378121 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:47:00.378121 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:47:00.378121 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:47:00.378121 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:47:00.378121 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:47:00.378121 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 09:47:00.378121 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 09:47:00.378121 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 02 09:47:00.378121 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:47:00.382378 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 404/812] 10.222.0.64 () {56 vars in 1111 bytes} [Tue Aug 2 09:47:00 2022] GET /identity/v3/services/network => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:47:00.383547 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d12920-f310-4d13-82d8-b089254c8d63 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:00.428829 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5085e87f-a8aa-44bf-a157-655a4f822901 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:00.481269 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4b62fc-f5ee-4a6a-8a93-95bc570f31f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:00.481904 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4b62fc-f5ee-4a6a-8a93-95bc570f31f0 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:00.482196 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4b62fc-f5ee-4a6a-8a93-95bc570f31f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:00.482475 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4b62fc-f5ee-4a6a-8a93-95bc570f31f0 None admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:00.483239 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4b62fc-f5ee-4a6a-8a93-95bc570f31f0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:47:00.483552 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4b62fc-f5ee-4a6a-8a93-95bc570f31f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:00.485025 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4b62fc-f5ee-4a6a-8a93-95bc570f31f0 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:00.493519 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 409/813] 10.222.0.64 () {56 vars in 1103 bytes} [Tue Aug 2 09:47:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:47:00.494438 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4b62fc-f5ee-4a6a-8a93-95bc570f31f0 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:00.517573 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d12920-f310-4d13-82d8-b089254c8d63 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:00.559741 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a01febe-4fd7-4746-990a-409350c89e2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:00.560466 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a01febe-4fd7-4746-990a-409350c89e2e None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:00.560760 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a01febe-4fd7-4746-990a-409350c89e2e None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:00.560940 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a01febe-4fd7-4746-990a-409350c89e2e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:00.574546 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:00.610109 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4b62fc-f5ee-4a6a-8a93-95bc570f31f0 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:00.624640 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:00.639495 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a01febe-4fd7-4746-990a-409350c89e2e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:00.641415 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a01febe-4fd7-4746-990a-409350c89e2e None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:00.649071 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66396d69-aa0f-4669-9da9-633a286e97cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:00.649757 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66396d69-aa0f-4669-9da9-633a286e97cd None admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:00.650045 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66396d69-aa0f-4669-9da9-633a286e97cd None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:00.650327 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66396d69-aa0f-4669-9da9-633a286e97cd None admin] PATH_INFO: `/v3/projects/390bd7c4f91c4465b0c7ab0f9e436c4d/users/6a371e3df3764a4cbb1b8368aa74bd65/roles/1000cf318fa14915b8091599eaead5de` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:00.652337 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66396d69-aa0f-4669-9da9-633a286e97cd None admin] RBAC: Authorizing `identity:create_grant(project_id=390bd7c4f91c4465b0c7ab0f9e436c4d, user_id=6a371e3df3764a4cbb1b8368aa74bd65, role_id=1000cf318fa14915b8091599eaead5de)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:00.653791 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66396d69-aa0f-4669-9da9-633a286e97cd None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:00.670493 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 410/814] 10.222.0.64 () {58 vars in 1439 bytes} [Tue Aug 2 09:47:00 2022] PUT /identity/v3/projects/390bd7c4f91c4465b0c7ab0f9e436c4d/users/6a371e3df3764a4cbb1b8368aa74bd65/roles/1000cf318fa14915b8091599eaead5de => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:47:00.673001 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66396d69-aa0f-4669-9da9-633a286e97cd None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:00.767690 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 405/815] 10.222.0.64 () {58 vars in 1282 bytes} [Tue Aug 2 09:47:00 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 02 09:47:00.767690 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a01febe-4fd7-4746-990a-409350c89e2e None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:00.822507 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:00.865698 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66396d69-aa0f-4669-9da9-633a286e97cd None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:00.894850 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a01febe-4fd7-4746-990a-409350c89e2e None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:00.913300 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67793936-2a63-46cc-b9d6-c9e1647428be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:00.913934 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67793936-2a63-46cc-b9d6-c9e1647428be None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:00.914223 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67793936-2a63-46cc-b9d6-c9e1647428be None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:00.914502 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67793936-2a63-46cc-b9d6-c9e1647428be None admin] PATH_INFO: `/v3/services` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:00.915203 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67793936-2a63-46cc-b9d6-c9e1647428be None admin] RBAC: Adding query filter params (name=network) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:47:00.915523 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67793936-2a63-46cc-b9d6-c9e1647428be None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:00.917506 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67793936-2a63-46cc-b9d6-c9e1647428be None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:00.924782 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 411/816] 10.222.0.64 () {56 vars in 1112 bytes} [Tue Aug 2 09:47:00 2022] GET /identity/v3/services?name=network => generated 124 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:47:00.925613 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67793936-2a63-46cc-b9d6-c9e1647428be None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:00.936557 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fef51c8-c7de-42fe-a263-4074ca94d600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:00.937947 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fef51c8-c7de-42fe-a263-4074ca94d600 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:00.938360 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fef51c8-c7de-42fe-a263-4074ca94d600 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:00.938737 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fef51c8-c7de-42fe-a263-4074ca94d600 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:00.949967 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fef51c8-c7de-42fe-a263-4074ca94d600 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:00.951567 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fef51c8-c7de-42fe-a263-4074ca94d600 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:00.967041 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:00.967229 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-7fef51c8-c7de-42fe-a263-4074ca94d600 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:47:00.967229 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:47:00.967229 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:47:00.967229 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:47:00.967229 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:47:00.967229 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:47:00.967229 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:47:00.967229 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:47:00.967229 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:47:00.967229 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:47:00.967229 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:47:00.967229 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:47:00.967229 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 09:47:00.967229 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 09:47:00.967229 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 09:47:00.968213 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 09:47:00.968213 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:47:00.968213 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:47:00.968213 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:47:00.968213 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:47:00.968213 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:47:00.968213 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:47:00.968213 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:47:00.968213 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:47:00.968213 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:47:00.968213 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:47:00.968213 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:47:00.968213 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:47:00.968213 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:47:00.968213 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:47:00.968213 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:47:00.968213 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:47:00.969265 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 09:47:00.969265 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 09:47:00.969265 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:47:00.969265 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:47:00.974046 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 406/817] 10.222.0.64 () {56 vars in 1108 bytes} [Tue Aug 2 09:47:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:47:00.975312 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fef51c8-c7de-42fe-a263-4074ca94d600 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:01.045988 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67793936-2a63-46cc-b9d6-c9e1647428be None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:01.090468 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad7b3c8-11e8-42b3-aa84-06cfaccbebe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:01.091299 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad7b3c8-11e8-42b3-aa84-06cfaccbebe8 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:01.091713 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad7b3c8-11e8-42b3-aa84-06cfaccbebe8 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:01.091999 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad7b3c8-11e8-42b3-aa84-06cfaccbebe8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:01.097323 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:01.106000 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fef51c8-c7de-42fe-a263-4074ca94d600 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:01.111330 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad7b3c8-11e8-42b3-aa84-06cfaccbebe8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:01.112863 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad7b3c8-11e8-42b3-aa84-06cfaccbebe8 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:01.134223 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-7ad7b3c8-11e8-42b3-aa84-06cfaccbebe8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:47:01.134223 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:47:01.134223 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:47:01.134223 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:47:01.134223 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:47:01.134223 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:47:01.134223 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:47:01.134223 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:47:01.134223 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:47:01.134223 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:47:01.134223 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:47:01.134223 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:47:01.134223 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 09:47:01.134223 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 09:47:01.134223 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 09:47:01.135225 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 09:47:01.135225 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:47:01.135225 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:47:01.135225 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:47:01.135225 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:47:01.135225 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:47:01.135225 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:47:01.135225 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:47:01.135225 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:47:01.135225 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:47:01.135225 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:47:01.135225 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:47:01.135225 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:47:01.135225 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:47:01.135225 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:47:01.135225 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:47:01.135225 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:47:01.136138 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 09:47:01.136138 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 09:47:01.136138 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:47:01.136138 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:47:01.136663 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 412/818] 10.222.0.64 () {56 vars in 1108 bytes} [Tue Aug 2 09:47:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:47:01.140282 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad7b3c8-11e8-42b3-aa84-06cfaccbebe8 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:01.150011 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbda9f85-0e88-4c52-8758-59c438cda87c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:01.150741 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbda9f85-0e88-4c52-8758-59c438cda87c None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:01.151097 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbda9f85-0e88-4c52-8758-59c438cda87c None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:01.151468 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbda9f85-0e88-4c52-8758-59c438cda87c None admin] PATH_INFO: `/v3/services` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:01.152235 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbda9f85-0e88-4c52-8758-59c438cda87c None admin] RBAC: Adding query filter params (type=network) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:47:01.152673 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbda9f85-0e88-4c52-8758-59c438cda87c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:01.154199 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbda9f85-0e88-4c52-8758-59c438cda87c None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:01.165162 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 407/819] 10.222.0.64 () {56 vars in 1112 bytes} [Tue Aug 2 09:47:00 2022] GET /identity/v3/services?type=network => generated 350 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:47:01.166262 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbda9f85-0e88-4c52-8758-59c438cda87c None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:01.260863 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:01.278024 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad7b3c8-11e8-42b3-aa84-06cfaccbebe8 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:01.300342 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbda9f85-0e88-4c52-8758-59c438cda87c None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:01.323600 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4079ed-7696-4ba8-9bb7-59147be05def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:01.324773 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4079ed-7696-4ba8-9bb7-59147be05def None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:01.325218 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4079ed-7696-4ba8-9bb7-59147be05def None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:01.325650 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4079ed-7696-4ba8-9bb7-59147be05def None admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:01.327194 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4079ed-7696-4ba8-9bb7-59147be05def None admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:47:01.327664 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4079ed-7696-4ba8-9bb7-59147be05def None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:01.329765 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4079ed-7696-4ba8-9bb7-59147be05def None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:01.338676 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8747d372-e0a1-41e5-be57-d83a6f70b506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:01.339527 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8747d372-e0a1-41e5-be57-d83a6f70b506 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:01.339896 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8747d372-e0a1-41e5-be57-d83a6f70b506 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:01.340263 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8747d372-e0a1-41e5-be57-d83a6f70b506 None admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:01.341046 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8747d372-e0a1-41e5-be57-d83a6f70b506 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:47:01.341424 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8747d372-e0a1-41e5-be57-d83a6f70b506 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:01.342997 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8747d372-e0a1-41e5-be57-d83a6f70b506 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:01.347761 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 413/820] 10.222.0.64 () {56 vars in 1109 bytes} [Tue Aug 2 09:47:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:47:01.348859 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4079ed-7696-4ba8-9bb7-59147be05def None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:01.366624 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 408/821] 10.222.0.64 () {56 vars in 1109 bytes} [Tue Aug 2 09:47:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:47:01.371300 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8747d372-e0a1-41e5-be57-d83a6f70b506 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:01.392173 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:01.492766 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4079ed-7696-4ba8-9bb7-59147be05def None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:01.494313 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8747d372-e0a1-41e5-be57-d83a6f70b506 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:01.534892 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93d2ee8-21a9-4165-bb87-0575bc732b5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:01.535657 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93d2ee8-21a9-4165-bb87-0575bc732b5d None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:01.535903 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93d2ee8-21a9-4165-bb87-0575bc732b5d None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:01.536167 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93d2ee8-21a9-4165-bb87-0575bc732b5d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:01.542040 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d4718f-1c76-42a7-9dd9-44fafead184f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:01.543091 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d4718f-1c76-42a7-9dd9-44fafead184f None admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:01.543669 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d4718f-1c76-42a7-9dd9-44fafead184f None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:01.544093 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93d2ee8-21a9-4165-bb87-0575bc732b5d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:01.544356 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d4718f-1c76-42a7-9dd9-44fafead184f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:01.545275 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d4718f-1c76-42a7-9dd9-44fafead184f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:01.545799 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93d2ee8-21a9-4165-bb87-0575bc732b5d None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:01.547238 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d4718f-1c76-42a7-9dd9-44fafead184f None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:01.558397 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-e93d2ee8-21a9-4165-bb87-0575bc732b5d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 02 09:47:01.558397 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:47:01.558397 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:47:01.558397 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:47:01.558397 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:47:01.558397 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:47:01.558397 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:47:01.558397 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:47:01.558397 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:47:01.558397 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:47:01.558397 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:47:01.558397 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:47:01.558397 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 09:47:01.558397 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 09:47:01.558397 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 09:47:01.558397 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 09:47:01.559421 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:47:01.559421 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:47:01.559421 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 09:47:01.559421 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 09:47:01.559421 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 09:47:01.559421 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 09:47:01.559421 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:47:01.559421 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:47:01.559421 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:47:01.559421 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:47:01.559421 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:47:01.559421 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:47:01.559421 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:47:01.559421 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:47:01.559421 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:47:01.559421 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:47:01.559421 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:47:01.559421 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:47:01.560888 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:47:01.560888 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:47:01.560888 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 09:47:01.560888 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 09:47:01.560888 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 09:47:01.560888 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 09:47:01.560888 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 09:47:01.560888 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 09:47:01.560888 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 02 09:47:01.560888 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:47:01.564694 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 409/822] 10.222.0.64 () {56 vars in 1117 bytes} [Tue Aug 2 09:47:01 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:47:01.565807 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93d2ee8-21a9-4165-bb87-0575bc732b5d None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:01.578499 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 414/823] 10.222.0.64 () {60 vars in 1144 bytes} [Tue Aug 2 09:47:01 2022] POST /identity/v3/endpoints => generated 337 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:47:01.579670 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d4718f-1c76-42a7-9dd9-44fafead184f None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:01.583788 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:01.686244 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:01.726481 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93d2ee8-21a9-4165-bb87-0575bc732b5d None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:01.735000 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d4718f-1c76-42a7-9dd9-44fafead184f None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:01.772912 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41754e46-243f-43db-9bcc-6e29f660612d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:01.773658 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41754e46-243f-43db-9bcc-6e29f660612d None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:01.774035 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41754e46-243f-43db-9bcc-6e29f660612d None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:01.774608 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41754e46-243f-43db-9bcc-6e29f660612d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:01.784240 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41754e46-243f-43db-9bcc-6e29f660612d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:01.785735 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41754e46-243f-43db-9bcc-6e29f660612d None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:01.787346 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6cd7da-5282-4c74-8b06-172b362c4b98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:01.787980 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6cd7da-5282-4c74-8b06-172b362c4b98 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:01.788287 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6cd7da-5282-4c74-8b06-172b362c4b98 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:01.788568 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6cd7da-5282-4c74-8b06-172b362c4b98 None admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:01.789392 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6cd7da-5282-4c74-8b06-172b362c4b98 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:47:01.789705 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6cd7da-5282-4c74-8b06-172b362c4b98 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:01.791186 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6cd7da-5282-4c74-8b06-172b362c4b98 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:01.801793 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-41754e46-243f-43db-9bcc-6e29f660612d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 09:47:01.801793 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:47:01.801793 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:47:01.801793 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:47:01.801793 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:47:01.801793 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:47:01.801793 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:47:01.801793 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:47:01.801793 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:47:01.801793 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:47:01.801793 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:47:01.801793 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:47:01.801793 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 09:47:01.801793 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 09:47:01.801793 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 09:47:01.802755 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 09:47:01.802755 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:47:01.802755 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:47:01.802755 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:47:01.802755 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:47:01.802755 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:47:01.802755 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:47:01.802755 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:47:01.802755 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:47:01.802755 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:47:01.802755 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:47:01.802755 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:47:01.802755 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:47:01.802755 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:47:01.802755 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:47:01.802755 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:47:01.802755 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:47:01.803684 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 09:47:01.803684 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 09:47:01.803684 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 09:47:01.803684 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 09:47:01.803684 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 09:47:01.803684 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 09:47:01.803684 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 09:47:01.803684 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:47:01.803684 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 410/824] 10.222.0.64 () {56 vars in 1111 bytes} [Tue Aug 2 09:47:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:47:01.807217 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88d39a2-6670-471a-afad-189939321953 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:01.807792 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88d39a2-6670-471a-afad-189939321953 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:01.808310 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88d39a2-6670-471a-afad-189939321953 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:01.810411 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 411/825] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:47:01 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:47:01.811126 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88d39a2-6670-471a-afad-189939321953 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:01.818594 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 415/826] 10.222.0.64 () {56 vars in 1149 bytes} [Tue Aug 2 09:47:01 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:47:01.820037 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce83cf96-117e-45a5-b632-2944b278235d None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:01.820472 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce83cf96-117e-45a5-b632-2944b278235d None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:01.820752 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce83cf96-117e-45a5-b632-2944b278235d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:01.842243 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-ce83cf96-117e-45a5-b632-2944b278235d None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:47:01.844011 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-ce83cf96-117e-45a5-b632-2944b278235d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:47:01.917338 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:01.932040 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 416/827] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:47:01 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 09:47:01.933220 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce83cf96-117e-45a5-b632-2944b278235d None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:01.955259 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88d39a2-6670-471a-afad-189939321953 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:01.992107 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e774f0-593f-4411-aa49-3ea1aad39dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:01.992768 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e774f0-593f-4411-aa49-3ea1aad39dc1 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:01.993058 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e774f0-593f-4411-aa49-3ea1aad39dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:01.993341 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e774f0-593f-4411-aa49-3ea1aad39dc1 None admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:01.994099 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e774f0-593f-4411-aa49-3ea1aad39dc1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:47:01.994409 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e774f0-593f-4411-aa49-3ea1aad39dc1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:01.995868 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e774f0-593f-4411-aa49-3ea1aad39dc1 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:02.003979 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:02.009754 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 412/828] 10.222.0.64 () {56 vars in 1148 bytes} [Tue Aug 2 09:47:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:47:02.010658 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e774f0-593f-4411-aa49-3ea1aad39dc1 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:02.045879 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce83cf96-117e-45a5-b632-2944b278235d None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:02.080098 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d9fa15-9ada-435f-8829-739c8a85fa24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:02.080687 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d9fa15-9ada-435f-8829-739c8a85fa24 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:02.080984 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d9fa15-9ada-435f-8829-739c8a85fa24 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:02.081260 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d9fa15-9ada-435f-8829-739c8a85fa24 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:02.091311 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d9fa15-9ada-435f-8829-739c8a85fa24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:02.092790 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d9fa15-9ada-435f-8829-739c8a85fa24 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:02.108403 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-30d9fa15-9ada-435f-8829-739c8a85fa24 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:47:02.108403 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:47:02.108403 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:47:02.108403 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:47:02.108403 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:47:02.108403 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:47:02.108403 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:47:02.108403 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:47:02.108403 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:47:02.108403 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:47:02.108403 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:47:02.108403 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:47:02.108403 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 09:47:02.108403 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 09:47:02.108403 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 09:47:02.109458 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 09:47:02.109458 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:47:02.109458 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:47:02.109458 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:47:02.109458 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:47:02.109458 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:47:02.109458 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:47:02.109458 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:47:02.109458 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:47:02.109458 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:47:02.109458 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:47:02.109458 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:47:02.109458 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:47:02.109458 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:47:02.109458 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:47:02.109458 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:47:02.109458 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:47:02.110526 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 09:47:02.110526 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 09:47:02.110526 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:47:02.110526 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:47:02.110526 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 417/829] 10.222.0.64 () {56 vars in 1108 bytes} [Tue Aug 2 09:47:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:47:02.110526 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d9fa15-9ada-435f-8829-739c8a85fa24 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:02.118231 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e774f0-593f-4411-aa49-3ea1aad39dc1 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:02.154043 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee44309f-f3b5-46be-b517-f7d54695a699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:02.154680 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee44309f-f3b5-46be-b517-f7d54695a699 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:02.154971 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee44309f-f3b5-46be-b517-f7d54695a699 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:02.155249 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee44309f-f3b5-46be-b517-f7d54695a699 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:02.156422 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee44309f-f3b5-46be-b517-f7d54695a699 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:02.157800 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee44309f-f3b5-46be-b517-f7d54695a699 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:02.158968 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:02.166874 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-ee44309f-f3b5-46be-b517-f7d54695a699 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 02 09:47:02.166874 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:47:02.166874 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 09:47:02.166874 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 09:47:02.166874 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 09:47:02.166874 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 09:47:02.166874 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 09:47:02.166874 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 09:47:02.166874 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 02 09:47:02.166874 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:47:02.166874 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 09:47:02.166874 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:47:02.166874 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:47:02.166874 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:47:02.166874 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:47:02.166874 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:47:02.166874 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:47:02.167895 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:47:02.167895 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:47:02.167895 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:47:02.167895 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:47:02.167895 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:47:02.167895 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:47:02.167895 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 09:47:02.167895 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 09:47:02.167895 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 09:47:02.167895 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 09:47:02.167895 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:47:02.167895 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:47:02.167895 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:47:02.167895 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:47:02.167895 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:47:02.167895 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:47:02.168782 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:47:02.168782 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:47:02.168782 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:47:02.168782 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:47:02.168782 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:47:02.168782 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:47:02.168782 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:47:02.168782 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:47:02.168782 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:47:02.168782 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:47:02.168782 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 09:47:02.168782 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 09:47:02.168782 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 02 09:47:02.168782 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:47:02.168782 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 413/830] 10.222.0.64 () {56 vars in 1114 bytes} [Tue Aug 2 09:47:02 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:47:02.169701 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee44309f-f3b5-46be-b517-f7d54695a699 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:02.228559 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d9fa15-9ada-435f-8829-739c8a85fa24 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:02.231202 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:02.264566 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4866d0-d4f3-4daf-98ee-4ddde8db7781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:02.265701 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4866d0-d4f3-4daf-98ee-4ddde8db7781 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:02.266003 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4866d0-d4f3-4daf-98ee-4ddde8db7781 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:02.266278 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4866d0-d4f3-4daf-98ee-4ddde8db7781 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:02.267087 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4866d0-d4f3-4daf-98ee-4ddde8db7781 None admin] RBAC: Adding query filter params (role.id=1000cf318fa14915b8091599eaead5de, scope.project.id=390bd7c4f91c4465b0c7ab0f9e436c4d, user.id=99655e1f9ca24472a86cd26b0dbd1fc2) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:47:02.267401 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4866d0-d4f3-4daf-98ee-4ddde8db7781 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:02.268872 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4866d0-d4f3-4daf-98ee-4ddde8db7781 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:02.272540 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee44309f-f3b5-46be-b517-f7d54695a699 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:02.274723 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 418/831] 10.222.0.64 () {56 vars in 1374 bytes} [Tue Aug 2 09:47:02 2022] GET /identity/v3/role_assignments?user.id=99655e1f9ca24472a86cd26b0dbd1fc2&scope.project.id=390bd7c4f91c4465b0c7ab0f9e436c4d&role.id=1000cf318fa14915b8091599eaead5de => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:47:02.275509 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4866d0-d4f3-4daf-98ee-4ddde8db7781 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:02.327346 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0f1c43-361e-48aa-bd92-c8744c1a2a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:02.328826 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0f1c43-361e-48aa-bd92-c8744c1a2a94 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:02.329752 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0f1c43-361e-48aa-bd92-c8744c1a2a94 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:02.330238 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0f1c43-361e-48aa-bd92-c8744c1a2a94 None admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:02.331580 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0f1c43-361e-48aa-bd92-c8744c1a2a94 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:47:02.332353 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0f1c43-361e-48aa-bd92-c8744c1a2a94 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:02.334258 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0f1c43-361e-48aa-bd92-c8744c1a2a94 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:02.364042 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 414/832] 10.222.0.64 () {56 vars in 1109 bytes} [Tue Aug 2 09:47:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:47:02.366174 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0f1c43-361e-48aa-bd92-c8744c1a2a94 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:02.411083 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4866d0-d4f3-4daf-98ee-4ddde8db7781 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:02.414120 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:02.452285 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c5c909-ff03-46ed-80d3-d3bcb337f8e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:02.453709 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c5c909-ff03-46ed-80d3-d3bcb337f8e3 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:02.453988 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c5c909-ff03-46ed-80d3-d3bcb337f8e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:02.454257 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c5c909-ff03-46ed-80d3-d3bcb337f8e3 None admin] PATH_INFO: `/v3/services` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:02.454944 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c5c909-ff03-46ed-80d3-d3bcb337f8e3 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:47:02.455249 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c5c909-ff03-46ed-80d3-d3bcb337f8e3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:02.456680 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c5c909-ff03-46ed-80d3-d3bcb337f8e3 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:02.467525 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 419/833] 10.222.0.64 () {56 vars in 1114 bytes} [Tue Aug 2 09:47:02 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:47:02.469053 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c5c909-ff03-46ed-80d3-d3bcb337f8e3 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:02.502651 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0f1c43-361e-48aa-bd92-c8744c1a2a94 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:02.533895 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:02.563134 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc8db3d-9e0b-488a-a19f-c778e0a87005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:02.565158 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc8db3d-9e0b-488a-a19f-c778e0a87005 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:02.567100 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc8db3d-9e0b-488a-a19f-c778e0a87005 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:02.567563 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc8db3d-9e0b-488a-a19f-c778e0a87005 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:02.579737 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc8db3d-9e0b-488a-a19f-c778e0a87005 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:02.582039 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc8db3d-9e0b-488a-a19f-c778e0a87005 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:02.595116 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-6dc8db3d-9e0b-488a-a19f-c778e0a87005 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 09:47:02.595116 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:47:02.595116 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:47:02.595116 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:47:02.595116 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:47:02.595116 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:47:02.595116 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:47:02.595116 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:47:02.595116 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:47:02.595116 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:47:02.595116 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:47:02.595116 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:47:02.595116 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 09:47:02.595116 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 09:47:02.595116 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 09:47:02.596077 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 09:47:02.596077 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:47:02.596077 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:47:02.596077 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:47:02.596077 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:47:02.596077 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:47:02.596077 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:47:02.596077 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:47:02.596077 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:47:02.596077 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:47:02.596077 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:47:02.596077 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:47:02.596077 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:47:02.596077 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:47:02.596077 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:47:02.596077 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:47:02.596077 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:47:02.597047 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 09:47:02.597047 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 09:47:02.597047 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 09:47:02.597047 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 09:47:02.597047 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 09:47:02.597047 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 09:47:02.597047 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 09:47:02.597047 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:47:02.597047 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 415/834] 10.222.0.64 () {56 vars in 1111 bytes} [Tue Aug 2 09:47:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 09:47:02.608368 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc8db3d-9e0b-488a-a19f-c778e0a87005 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:02.614331 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c5c909-ff03-46ed-80d3-d3bcb337f8e3 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:02.654795 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f68126-aed3-4a2f-b0f3-80865ed69072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:02.655428 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f68126-aed3-4a2f-b0f3-80865ed69072 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:02.655719 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f68126-aed3-4a2f-b0f3-80865ed69072 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:02.656059 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f68126-aed3-4a2f-b0f3-80865ed69072 None admin] PATH_INFO: `/v3/services` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:02.656784 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f68126-aed3-4a2f-b0f3-80865ed69072 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:47:02.657096 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f68126-aed3-4a2f-b0f3-80865ed69072 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:02.658478 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f68126-aed3-4a2f-b0f3-80865ed69072 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:02.663816 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 420/835] 10.222.0.64 () {56 vars in 1114 bytes} [Tue Aug 2 09:47:02 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:47:02.756679 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc8db3d-9e0b-488a-a19f-c778e0a87005 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:02.800949 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcafe8b-6560-4ab6-bed8-621906f6bf0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:02.801784 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdc6ac4-b978-4c5a-9ce3-7053a75b1316 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:02.802255 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdc6ac4-b978-4c5a-9ce3-7053a75b1316 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:02.802339 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdc6ac4-b978-4c5a-9ce3-7053a75b1316 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:02.802826 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcafe8b-6560-4ab6-bed8-621906f6bf0d None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:02.803318 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcafe8b-6560-4ab6-bed8-621906f6bf0d None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:02.803797 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcafe8b-6560-4ab6-bed8-621906f6bf0d None admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:02.804871 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcafe8b-6560-4ab6-bed8-621906f6bf0d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:47:02.805352 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 421/836] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:47:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:47:02.805633 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcafe8b-6560-4ab6-bed8-621906f6bf0d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:02.807403 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcafe8b-6560-4ab6-bed8-621906f6bf0d None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:02.810118 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e03bc43-19a6-467e-adcb-3ea3a677a9d1 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:02.810421 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e03bc43-19a6-467e-adcb-3ea3a677a9d1 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:02.810688 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e03bc43-19a6-467e-adcb-3ea3a677a9d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:02.826173 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 416/837] 10.222.0.64 () {56 vars in 1148 bytes} [Tue Aug 2 09:47:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:47:02.833313 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcafe8b-6560-4ab6-bed8-621906f6bf0d None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:02.839142 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-6e03bc43-19a6-467e-adcb-3ea3a677a9d1 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:47:02.840809 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-6e03bc43-19a6-467e-adcb-3ea3a677a9d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:47:02.874382 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:02.890926 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:02.936106 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 422/838] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:47:02 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 02 09:47:02.959317 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e03bc43-19a6-467e-adcb-3ea3a677a9d1 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:02.963693 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcafe8b-6560-4ab6-bed8-621906f6bf0d None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:03.000988 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a027b486-9d8c-4a21-8495-b086b87be563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:03.001662 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a027b486-9d8c-4a21-8495-b086b87be563 None admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:03.001948 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a027b486-9d8c-4a21-8495-b086b87be563 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:03.002227 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a027b486-9d8c-4a21-8495-b086b87be563 None admin] PATH_INFO: `/v3/projects/390bd7c4f91c4465b0c7ab0f9e436c4d/users/5338bc8c594e4bc9bc41019226bf8d87/roles/5704af09562a457fa35364e8af9481f1` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:03.018447 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a027b486-9d8c-4a21-8495-b086b87be563 None admin] RBAC: Authorizing `identity:create_grant(project_id=390bd7c4f91c4465b0c7ab0f9e436c4d, user_id=5338bc8c594e4bc9bc41019226bf8d87, role_id=5704af09562a457fa35364e8af9481f1)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:03.020042 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a027b486-9d8c-4a21-8495-b086b87be563 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:03.044277 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 417/839] 10.222.0.64 () {58 vars in 1439 bytes} [Tue Aug 2 09:47:02 2022] PUT /identity/v3/projects/390bd7c4f91c4465b0c7ab0f9e436c4d/users/5338bc8c594e4bc9bc41019226bf8d87/roles/5704af09562a457fa35364e8af9481f1 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:47:03.086224 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e03bc43-19a6-467e-adcb-3ea3a677a9d1 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:03.133245 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e06d7f1-b1e1-47f3-b1e3-77c918f372a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:03.133967 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e06d7f1-b1e1-47f3-b1e3-77c918f372a2 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:03.134255 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e06d7f1-b1e1-47f3-b1e3-77c918f372a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:03.134534 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e06d7f1-b1e1-47f3-b1e3-77c918f372a2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:03.148020 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e06d7f1-b1e1-47f3-b1e3-77c918f372a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 09:47:03.153229 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-8e06d7f1-b1e1-47f3-b1e3-77c918f372a2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 09:47:03.153229 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:47:03.153229 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:47:03.153229 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:47:03.153229 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:47:03.153229 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:47:03.153229 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:47:03.153229 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:47:03.153229 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:47:03.153229 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:47:03.153229 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:47:03.153229 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:47:03.153229 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 09:47:03.153229 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 09:47:03.153229 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 09:47:03.153229 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 09:47:03.154254 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 09:47:03.154254 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 09:47:03.154254 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 09:47:03.154254 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 09:47:03.154254 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:47:03.154254 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:47:03.154254 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:47:03.154254 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:47:03.154254 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:47:03.154254 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:47:03.154254 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:47:03.154254 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:47:03.154254 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:47:03.154254 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:47:03.154254 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:47:03.154254 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:47:03.155586 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:47:03.155586 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:47:03.155586 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:47:03.155586 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:47:03.155586 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 09:47:03.155586 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 09:47:03.155586 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 09:47:03.155586 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 09:47:03.155586 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 09:47:03.155586 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 09:47:03.155586 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 09:47:03.155586 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:47:03.155586 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 423/840] 10.222.0.64 () {56 vars in 1096 bytes} [Tue Aug 2 09:47:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 09:47:03.169480 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a027b486-9d8c-4a21-8495-b086b87be563 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:03.276188 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:03.288028 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a027b486-9d8c-4a21-8495-b086b87be563 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:03.329217 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040ccab7-a6b1-4226-a584-56870ef2e885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:03.329846 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040ccab7-a6b1-4226-a584-56870ef2e885 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:03.330131 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040ccab7-a6b1-4226-a584-56870ef2e885 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:03.330413 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040ccab7-a6b1-4226-a584-56870ef2e885 None admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:03.331146 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040ccab7-a6b1-4226-a584-56870ef2e885 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:47:03.331456 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040ccab7-a6b1-4226-a584-56870ef2e885 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:03.332884 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040ccab7-a6b1-4226-a584-56870ef2e885 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:03.342066 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 418/841] 10.222.0.64 () {56 vars in 1099 bytes} [Tue Aug 2 09:47:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:47:03.348457 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e06d7f1-b1e1-47f3-b1e3-77c918f372a2 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:03.376794 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:03.458254 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e06d7f1-b1e1-47f3-b1e3-77c918f372a2 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:03.494197 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc265ec-097f-40b7-b9f5-4641598765b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:03.494855 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc265ec-097f-40b7-b9f5-4641598765b0 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:03.495137 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc265ec-097f-40b7-b9f5-4641598765b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:03.495413 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc265ec-097f-40b7-b9f5-4641598765b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:03.506803 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc265ec-097f-40b7-b9f5-4641598765b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:03.508282 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc265ec-097f-40b7-b9f5-4641598765b0 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:03.524351 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-9fc265ec-097f-40b7-b9f5-4641598765b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:47:03.524351 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:47:03.524351 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:47:03.524351 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:47:03.524351 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:47:03.524351 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:47:03.524351 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:47:03.524351 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:47:03.524351 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:47:03.524351 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:47:03.524351 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:47:03.524351 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:47:03.524351 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 09:47:03.524351 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 09:47:03.524351 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 09:47:03.525410 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 09:47:03.525410 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:47:03.525410 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:47:03.525410 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:47:03.525410 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:47:03.525410 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:47:03.525410 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:47:03.525410 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:47:03.525410 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:47:03.525410 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:47:03.525410 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:47:03.525410 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:47:03.525410 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:47:03.525410 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:47:03.525410 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:47:03.525410 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:47:03.525410 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:47:03.526753 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 09:47:03.526753 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 09:47:03.526753 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:47:03.526753 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:47:03.526753 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 424/842] 10.222.0.64 () {56 vars in 1108 bytes} [Tue Aug 2 09:47:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:47:03.533367 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040ccab7-a6b1-4226-a584-56870ef2e885 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:03.646036 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040ccab7-a6b1-4226-a584-56870ef2e885 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:03.684980 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2347522b-e955-4f40-a452-5ad7074e54df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:03.685603 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2347522b-e955-4f40-a452-5ad7074e54df None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:03.685890 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2347522b-e955-4f40-a452-5ad7074e54df None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:03.686166 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2347522b-e955-4f40-a452-5ad7074e54df None admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:03.686865 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2347522b-e955-4f40-a452-5ad7074e54df None admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:47:03.687691 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2347522b-e955-4f40-a452-5ad7074e54df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:03.689130 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2347522b-e955-4f40-a452-5ad7074e54df None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:03.701470 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 419/843] 10.222.0.64 () {56 vars in 1109 bytes} [Tue Aug 2 09:47:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:47:03.709328 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc265ec-097f-40b7-b9f5-4641598765b0 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:03.788469 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:03.820364 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc265ec-097f-40b7-b9f5-4641598765b0 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:03.864907 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96449f01-8d68-4e88-89b9-6d43f6dd7aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:03.865575 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96449f01-8d68-4e88-89b9-6d43f6dd7aab None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:03.865863 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96449f01-8d68-4e88-89b9-6d43f6dd7aab None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:03.866141 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96449f01-8d68-4e88-89b9-6d43f6dd7aab None admin] PATH_INFO: `/v3/users/swift` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:03.875765 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96449f01-8d68-4e88-89b9-6d43f6dd7aab None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:03.877252 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96449f01-8d68-4e88-89b9-6d43f6dd7aab None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:03.894962 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-96449f01-8d68-4e88-89b9-6d43f6dd7aab None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 02 09:47:03.894962 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:47:03.894962 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:47:03.894962 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:47:03.894962 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:47:03.894962 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:47:03.894962 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:47:03.894962 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:47:03.894962 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:47:03.894962 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:47:03.894962 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:47:03.894962 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:47:03.894962 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 09:47:03.894962 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 09:47:03.894962 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 09:47:03.894962 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 09:47:03.895938 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:47:03.895938 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:47:03.895938 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 09:47:03.895938 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 09:47:03.895938 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 09:47:03.895938 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 09:47:03.895938 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:47:03.895938 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:47:03.895938 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:47:03.895938 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:47:03.895938 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:47:03.895938 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:47:03.895938 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:47:03.895938 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:47:03.895938 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:47:03.895938 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:47:03.895938 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:47:03.895938 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:47:03.897022 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:47:03.897022 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:47:03.897022 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 09:47:03.897022 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 09:47:03.897022 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 09:47:03.897022 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 09:47:03.897022 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 09:47:03.897022 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 09:47:03.897022 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 02 09:47:03.897022 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:47:03.897022 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 425/844] 10.222.0.64 () {56 vars in 1096 bytes} [Tue Aug 2 09:47:03 2022] GET /identity/v3/users/swift => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:47:03.902171 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2347522b-e955-4f40-a452-5ad7074e54df None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:03.932974 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:04.038440 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2347522b-e955-4f40-a452-5ad7074e54df None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:04.074964 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1427a5e-ecee-4529-ad9f-684a60bf3d57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:04.075595 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1427a5e-ecee-4529-ad9f-684a60bf3d57 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:04.075926 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1427a5e-ecee-4529-ad9f-684a60bf3d57 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:04.076206 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1427a5e-ecee-4529-ad9f-684a60bf3d57 None admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:04.077053 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1427a5e-ecee-4529-ad9f-684a60bf3d57 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:47:04.077368 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1427a5e-ecee-4529-ad9f-684a60bf3d57 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:04.078879 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1427a5e-ecee-4529-ad9f-684a60bf3d57 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:04.098700 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 420/845] 10.222.0.64 () {56 vars in 1135 bytes} [Tue Aug 2 09:47:03 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:47:04.104471 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96449f01-8d68-4e88-89b9-6d43f6dd7aab None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:04.213165 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96449f01-8d68-4e88-89b9-6d43f6dd7aab None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:04.251459 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1574e80e-5683-458d-825d-0aeb95b5fe27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:04.252089 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1574e80e-5683-458d-825d-0aeb95b5fe27 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:04.252411 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1574e80e-5683-458d-825d-0aeb95b5fe27 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:04.252691 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1574e80e-5683-458d-825d-0aeb95b5fe27 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:04.255459 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:04.326262 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1574e80e-5683-458d-825d-0aeb95b5fe27 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:04.327909 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1574e80e-5683-458d-825d-0aeb95b5fe27 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:04.434888 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 426/846] 10.222.0.64 () {58 vars in 1282 bytes} [Tue Aug 2 09:47:04 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 02 09:47:04.441397 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1427a5e-ecee-4529-ad9f-684a60bf3d57 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:04.524680 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:04.551746 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e13d55-0b71-4330-92ae-a96f3a33fe38 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:04.552090 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e13d55-0b71-4330-92ae-a96f3a33fe38 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:04.552345 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e13d55-0b71-4330-92ae-a96f3a33fe38 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:04.553825 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 427/847] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:47:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:47:04.559181 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedfaad0-7117-4890-a58e-b2c35155c4f9 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:04.559477 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedfaad0-7117-4890-a58e-b2c35155c4f9 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:04.559674 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1427a5e-ecee-4529-ad9f-684a60bf3d57 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:04.559778 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedfaad0-7117-4890-a58e-b2c35155c4f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:04.577540 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-aedfaad0-7117-4890-a58e-b2c35155c4f9 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:47:04.579281 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-aedfaad0-7117-4890-a58e-b2c35155c4f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:47:04.601792 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199cf662-d436-4e58-b645-a83545d5424f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:04.602879 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199cf662-d436-4e58-b645-a83545d5424f None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:04.603167 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199cf662-d436-4e58-b645-a83545d5424f None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:04.603716 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199cf662-d436-4e58-b645-a83545d5424f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:04.608861 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:04.617937 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199cf662-d436-4e58-b645-a83545d5424f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:04.619421 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199cf662-d436-4e58-b645-a83545d5424f None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:04.639311 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-199cf662-d436-4e58-b645-a83545d5424f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:47:04.639311 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:47:04.639311 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:47:04.639311 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:47:04.639311 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:47:04.639311 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:47:04.639311 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:47:04.639311 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:47:04.639311 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:47:04.639311 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:47:04.639311 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:47:04.639311 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:47:04.639311 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 09:47:04.639311 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 09:47:04.639311 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 09:47:04.640350 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 09:47:04.640350 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:47:04.640350 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:47:04.640350 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:47:04.640350 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:47:04.640350 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:47:04.640350 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:47:04.640350 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:47:04.640350 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:47:04.640350 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:47:04.640350 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:47:04.640350 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:47:04.640350 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:47:04.640350 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:47:04.640350 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:47:04.640350 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:47:04.640350 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:47:04.641310 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 09:47:04.641310 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 09:47:04.641310 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:47:04.641310 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:47:04.641310 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 421/848] 10.222.0.64 () {56 vars in 1108 bytes} [Tue Aug 2 09:47:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:47:04.646662 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199cf662-d436-4e58-b645-a83545d5424f None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:04.667741 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 428/849] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:47:04 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 09:47:04.690636 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedfaad0-7117-4890-a58e-b2c35155c4f9 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:04.776162 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199cf662-d436-4e58-b645-a83545d5424f None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:04.802224 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:04.813880 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49703eaa-9406-4f54-b987-63ec97751405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:04.814513 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49703eaa-9406-4f54-b987-63ec97751405 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:04.814803 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49703eaa-9406-4f54-b987-63ec97751405 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:04.815084 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49703eaa-9406-4f54-b987-63ec97751405 None admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:04.815792 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49703eaa-9406-4f54-b987-63ec97751405 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:47:04.816103 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49703eaa-9406-4f54-b987-63ec97751405 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:04.816431 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedfaad0-7117-4890-a58e-b2c35155c4f9 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:04.817552 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49703eaa-9406-4f54-b987-63ec97751405 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:04.830648 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 422/850] 10.222.0.64 () {56 vars in 1109 bytes} [Tue Aug 2 09:47:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:47:04.831772 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de02c3a8-3291-4604-b4db-a917406adcbf None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:04.832063 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de02c3a8-3291-4604-b4db-a917406adcbf None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:04.832374 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de02c3a8-3291-4604-b4db-a917406adcbf None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:04.833741 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 423/851] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:47:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:47:04.838386 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de02c3a8-3291-4604-b4db-a917406adcbf None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:04.863694 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0fd8d3-3871-40cb-bae4-9521c573fec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:04.864416 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0fd8d3-3871-40cb-bae4-9521c573fec7 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:04.864699 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0fd8d3-3871-40cb-bae4-9521c573fec7 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:04.865098 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0fd8d3-3871-40cb-bae4-9521c573fec7 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:04.865737 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0fd8d3-3871-40cb-bae4-9521c573fec7 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:04.867120 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0fd8d3-3871-40cb-bae4-9521c573fec7 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:04.876526 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-bb0fd8d3-3871-40cb-bae4-9521c573fec7 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 02 09:47:04.876526 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:47:04.876526 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 09:47:04.876526 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 09:47:04.876526 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 09:47:04.876526 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 09:47:04.876526 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 09:47:04.876526 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 09:47:04.876526 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 02 09:47:04.876526 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:47:04.876526 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 09:47:04.876526 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:47:04.876526 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:47:04.876526 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:47:04.876526 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:47:04.876526 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:47:04.877525 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:47:04.877525 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:47:04.877525 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:47:04.877525 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:47:04.877525 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:47:04.877525 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:47:04.877525 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:47:04.877525 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 09:47:04.877525 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 09:47:04.877525 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 09:47:04.877525 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 09:47:04.877525 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:47:04.877525 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:47:04.877525 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:47:04.877525 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:47:04.877525 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:47:04.878415 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:47:04.878415 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:47:04.878415 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:47:04.878415 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:47:04.878415 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:47:04.878415 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:47:04.878415 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:47:04.878415 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:47:04.878415 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:47:04.878415 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:47:04.878415 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:47:04.878415 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 09:47:04.878415 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 09:47:04.878415 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 02 09:47:04.878415 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:47:04.878415 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 429/852] 10.222.0.64 () {56 vars in 1132 bytes} [Tue Aug 2 09:47:04 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 09:47:04.879370 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93ba6b3-7f66-49b7-96dc-2ceee8dfe3eb None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:04.879370 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93ba6b3-7f66-49b7-96dc-2ceee8dfe3eb None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:04.880445 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93ba6b3-7f66-49b7-96dc-2ceee8dfe3eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:04.900522 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-b93ba6b3-7f66-49b7-96dc-2ceee8dfe3eb None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:47:04.901826 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-b93ba6b3-7f66-49b7-96dc-2ceee8dfe3eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:47:04.927199 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:04.974676 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 430/853] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:47:04 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 09:47:04.975897 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93ba6b3-7f66-49b7-96dc-2ceee8dfe3eb None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:04.987024 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de02c3a8-3291-4604-b4db-a917406adcbf None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:05.027144 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226c35d4-ba8c-4e9c-9896-481a72ea863b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:05.027708 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226c35d4-ba8c-4e9c-9896-481a72ea863b None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:05.027999 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226c35d4-ba8c-4e9c-9896-481a72ea863b None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:05.028295 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226c35d4-ba8c-4e9c-9896-481a72ea863b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:05.035406 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226c35d4-ba8c-4e9c-9896-481a72ea863b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:05.036902 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226c35d4-ba8c-4e9c-9896-481a72ea863b None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:05.049016 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-226c35d4-ba8c-4e9c-9896-481a72ea863b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 09:47:05.049016 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:47:05.049016 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:47:05.049016 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:47:05.049016 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:47:05.049016 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:47:05.049016 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:47:05.049016 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:47:05.049016 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:47:05.049016 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:47:05.049016 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:47:05.049016 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:47:05.049016 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 09:47:05.049016 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 09:47:05.049016 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 09:47:05.049994 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 09:47:05.049994 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:47:05.049994 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:47:05.049994 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:47:05.049994 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:47:05.049994 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:47:05.049994 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:47:05.049994 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:47:05.049994 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:47:05.049994 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:47:05.049994 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:47:05.049994 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:47:05.049994 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:47:05.049994 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:47:05.049994 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:47:05.049994 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:47:05.049994 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:47:05.050911 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 09:47:05.050911 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 09:47:05.050911 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 09:47:05.050911 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 09:47:05.050911 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 09:47:05.050911 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 09:47:05.050911 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 09:47:05.050911 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:47:05.054991 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 424/854] 10.222.0.64 () {56 vars in 1111 bytes} [Tue Aug 2 09:47:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:47:05.056018 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226c35d4-ba8c-4e9c-9896-481a72ea863b None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:05.091963 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93ba6b3-7f66-49b7-96dc-2ceee8dfe3eb None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:05.104436 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:05.136566 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf13c26-7e59-439e-8f8c-e79ce1b25622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:05.137194 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf13c26-7e59-439e-8f8c-e79ce1b25622 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:05.137483 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf13c26-7e59-439e-8f8c-e79ce1b25622 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:05.137763 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf13c26-7e59-439e-8f8c-e79ce1b25622 None admin] PATH_INFO: `/v3/services` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:05.138461 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf13c26-7e59-439e-8f8c-e79ce1b25622 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:47:05.138770 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf13c26-7e59-439e-8f8c-e79ce1b25622 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:05.140228 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf13c26-7e59-439e-8f8c-e79ce1b25622 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:05.150321 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 431/855] 10.222.0.64 () {56 vars in 1126 bytes} [Tue Aug 2 09:47:04 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:47:05.150321 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf13c26-7e59-439e-8f8c-e79ce1b25622 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:05.174800 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:05.180116 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226c35d4-ba8c-4e9c-9896-481a72ea863b None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:05.220678 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7686d0f3-ddbc-47fd-a737-3339cc75b035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:05.221573 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7686d0f3-ddbc-47fd-a737-3339cc75b035 None admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:05.221684 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7686d0f3-ddbc-47fd-a737-3339cc75b035 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:05.221939 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7686d0f3-ddbc-47fd-a737-3339cc75b035 None admin] PATH_INFO: `/v3/services` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:05.222673 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7686d0f3-ddbc-47fd-a737-3339cc75b035 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:05.224043 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7686d0f3-ddbc-47fd-a737-3339cc75b035 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:05.240847 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 425/856] 10.222.0.64 () {60 vars in 1141 bytes} [Tue Aug 2 09:47:05 2022] POST /identity/v3/services => generated 251 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:47:05.241704 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7686d0f3-ddbc-47fd-a737-3339cc75b035 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:05.276363 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf13c26-7e59-439e-8f8c-e79ce1b25622 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:05.328030 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e5da85-0f30-4d37-aef4-b87f64d7d52a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:05.328810 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e5da85-0f30-4d37-aef4-b87f64d7d52a None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:05.329091 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e5da85-0f30-4d37-aef4-b87f64d7d52a None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:05.329363 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e5da85-0f30-4d37-aef4-b87f64d7d52a None admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:05.330145 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e5da85-0f30-4d37-aef4-b87f64d7d52a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:47:05.330508 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e5da85-0f30-4d37-aef4-b87f64d7d52a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:05.332797 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e5da85-0f30-4d37-aef4-b87f64d7d52a None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:05.347154 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 432/857] 10.222.0.64 () {56 vars in 1148 bytes} [Tue Aug 2 09:47:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:47:05.348598 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82ed78f-a8fc-41f9-9850-39c2848e728e None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:05.348890 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82ed78f-a8fc-41f9-9850-39c2848e728e None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:05.349427 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82ed78f-a8fc-41f9-9850-39c2848e728e None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:05.356303 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 433/858] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:47:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:47:05.356303 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82ed78f-a8fc-41f9-9850-39c2848e728e None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:05.422890 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7686d0f3-ddbc-47fd-a737-3339cc75b035 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:05.425349 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:05.455182 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:05.465621 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1031fc-748c-4e5c-a9a3-f32eff46f703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:05.466248 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1031fc-748c-4e5c-a9a3-f32eff46f703 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:05.466530 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1031fc-748c-4e5c-a9a3-f32eff46f703 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:05.466803 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1031fc-748c-4e5c-a9a3-f32eff46f703 None admin] PATH_INFO: `/v3/services` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:05.467496 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1031fc-748c-4e5c-a9a3-f32eff46f703 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:47:05.467798 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1031fc-748c-4e5c-a9a3-f32eff46f703 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:05.469748 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1031fc-748c-4e5c-a9a3-f32eff46f703 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:05.480527 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 426/859] 10.222.0.64 () {56 vars in 1126 bytes} [Tue Aug 2 09:47:05 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:47:05.480527 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a879cbd-8ca4-414a-a316-b2c34cb09714 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:05.480527 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a879cbd-8ca4-414a-a316-b2c34cb09714 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:05.480527 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a879cbd-8ca4-414a-a316-b2c34cb09714 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:05.490231 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82ed78f-a8fc-41f9-9850-39c2848e728e None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:05.498369 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-8a879cbd-8ca4-414a-a316-b2c34cb09714 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:47:05.500112 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-8a879cbd-8ca4-414a-a316-b2c34cb09714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:47:05.531412 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15098a96-cbc3-4225-8d4f-05dbb5012472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:05.532363 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15098a96-cbc3-4225-8d4f-05dbb5012472 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:05.532815 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15098a96-cbc3-4225-8d4f-05dbb5012472 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:05.533093 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15098a96-cbc3-4225-8d4f-05dbb5012472 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:05.533931 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15098a96-cbc3-4225-8d4f-05dbb5012472 None admin] RBAC: Adding query filter params (role.id=1000cf318fa14915b8091599eaead5de, scope.project.id=390bd7c4f91c4465b0c7ab0f9e436c4d, user.id=6a371e3df3764a4cbb1b8368aa74bd65) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:47:05.534276 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15098a96-cbc3-4225-8d4f-05dbb5012472 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:05.535761 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15098a96-cbc3-4225-8d4f-05dbb5012472 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:05.543112 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 434/860] 10.222.0.64 () {56 vars in 1374 bytes} [Tue Aug 2 09:47:05 2022] GET /identity/v3/role_assignments?user.id=6a371e3df3764a4cbb1b8368aa74bd65&scope.project.id=390bd7c4f91c4465b0c7ab0f9e436c4d&role.id=1000cf318fa14915b8091599eaead5de => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:47:05.585146 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 427/861] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:47:05 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 09:47:05.609127 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15098a96-cbc3-4225-8d4f-05dbb5012472 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:05.745150 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15098a96-cbc3-4225-8d4f-05dbb5012472 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:05.784828 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93838602-2ae4-48b0-a8bd-c5e57eb7ec07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:05.785482 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93838602-2ae4-48b0-a8bd-c5e57eb7ec07 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:05.785771 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93838602-2ae4-48b0-a8bd-c5e57eb7ec07 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:05.786044 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93838602-2ae4-48b0-a8bd-c5e57eb7ec07 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:05.797940 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93838602-2ae4-48b0-a8bd-c5e57eb7ec07 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 09:47:05.804183 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-93838602-2ae4-48b0-a8bd-c5e57eb7ec07 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 09:47:05.804183 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:47:05.804183 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:47:05.804183 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:47:05.804183 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:47:05.804183 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:47:05.804183 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:47:05.804183 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:47:05.804183 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:47:05.804183 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:47:05.804183 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:47:05.804183 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:47:05.804183 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 09:47:05.804183 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 09:47:05.804183 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 09:47:05.804183 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 09:47:05.805297 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 09:47:05.805297 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 09:47:05.805297 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 09:47:05.805297 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 09:47:05.805297 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:47:05.805297 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:47:05.805297 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:47:05.805297 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:47:05.805297 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:47:05.805297 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:47:05.805297 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:47:05.805297 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:47:05.805297 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:47:05.805297 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:47:05.805297 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:47:05.805297 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:47:05.806143 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:47:05.806143 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:47:05.806143 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:47:05.806143 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:47:05.806143 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 09:47:05.806143 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 09:47:05.806143 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 09:47:05.806143 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 09:47:05.806143 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 09:47:05.806143 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 09:47:05.806143 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 09:47:05.806143 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:47:05.806143 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 435/862] 10.222.0.64 () {56 vars in 1102 bytes} [Tue Aug 2 09:47:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:47:05.813625 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a879cbd-8ca4-414a-a316-b2c34cb09714 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:05.920378 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:05.925166 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a879cbd-8ca4-414a-a316-b2c34cb09714 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:05.961654 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce78568-a404-4d84-9e99-e7a6a11f84bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:05.962335 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce78568-a404-4d84-9e99-e7a6a11f84bc None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:05.962623 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce78568-a404-4d84-9e99-e7a6a11f84bc None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:05.962900 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce78568-a404-4d84-9e99-e7a6a11f84bc None admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:05.963631 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce78568-a404-4d84-9e99-e7a6a11f84bc None admin] RBAC: Adding query filter params (name=service) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:47:05.963939 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce78568-a404-4d84-9e99-e7a6a11f84bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:05.965394 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce78568-a404-4d84-9e99-e7a6a11f84bc None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:05.974695 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 428/863] 10.222.0.64 () {56 vars in 1103 bytes} [Tue Aug 2 09:47:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:47:05.981335 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93838602-2ae4-48b0-a8bd-c5e57eb7ec07 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:05.984850 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:06.093898 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93838602-2ae4-48b0-a8bd-c5e57eb7ec07 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:06.134734 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da824c5-501b-4a5f-a1b2-5cabf6759ccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:06.135429 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da824c5-501b-4a5f-a1b2-5cabf6759ccc None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:06.135719 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da824c5-501b-4a5f-a1b2-5cabf6759ccc None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:06.136000 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da824c5-501b-4a5f-a1b2-5cabf6759ccc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:06.147315 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da824c5-501b-4a5f-a1b2-5cabf6759ccc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:06.148942 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da824c5-501b-4a5f-a1b2-5cabf6759ccc None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:06.163451 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-3da824c5-501b-4a5f-a1b2-5cabf6759ccc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:47:06.163451 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:47:06.163451 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:47:06.163451 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:47:06.163451 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:47:06.163451 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:47:06.163451 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:47:06.163451 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:47:06.163451 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:47:06.163451 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:47:06.163451 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:47:06.163451 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:47:06.163451 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 09:47:06.163451 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 09:47:06.163451 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 09:47:06.164380 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 09:47:06.164380 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:47:06.164380 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:47:06.164380 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:47:06.164380 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:47:06.164380 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:47:06.164380 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:47:06.164380 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:47:06.164380 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:47:06.164380 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:47:06.164380 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:47:06.164380 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:47:06.164380 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:47:06.164380 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:47:06.164380 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:47:06.164380 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:47:06.164380 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:47:06.165371 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 09:47:06.165371 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 09:47:06.165371 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:47:06.165371 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:47:06.165371 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 436/864] 10.222.0.64 () {56 vars in 1108 bytes} [Tue Aug 2 09:47:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:47:06.172377 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce78568-a404-4d84-9e99-e7a6a11f84bc None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:06.274645 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce78568-a404-4d84-9e99-e7a6a11f84bc None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:06.310040 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86befffa-ca15-46f5-914b-721fccfc669f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:06.311015 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86befffa-ca15-46f5-914b-721fccfc669f None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:06.311497 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86befffa-ca15-46f5-914b-721fccfc669f None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:06.312672 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86befffa-ca15-46f5-914b-721fccfc669f None admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:06.313626 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86befffa-ca15-46f5-914b-721fccfc669f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:47:06.314126 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86befffa-ca15-46f5-914b-721fccfc669f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:06.315748 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86befffa-ca15-46f5-914b-721fccfc669f None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:06.328657 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 429/865] 10.222.0.64 () {56 vars in 1109 bytes} [Tue Aug 2 09:47:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:47:06.334596 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da824c5-501b-4a5f-a1b2-5cabf6759ccc None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:06.399937 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:06.450109 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da824c5-501b-4a5f-a1b2-5cabf6759ccc None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:06.485652 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b101b82a-832c-4ca5-b85d-7581590565fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:06.486300 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b101b82a-832c-4ca5-b85d-7581590565fb None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:06.486589 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b101b82a-832c-4ca5-b85d-7581590565fb None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:06.486869 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b101b82a-832c-4ca5-b85d-7581590565fb None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:06.494585 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b101b82a-832c-4ca5-b85d-7581590565fb None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:06.496113 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b101b82a-832c-4ca5-b85d-7581590565fb None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:06.508828 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-b101b82a-832c-4ca5-b85d-7581590565fb None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 02 09:47:06.508828 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:47:06.508828 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:47:06.508828 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:47:06.508828 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:47:06.508828 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:47:06.508828 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:47:06.508828 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:47:06.508828 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:47:06.508828 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:47:06.508828 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:47:06.508828 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:47:06.508828 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 09:47:06.508828 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 09:47:06.508828 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 09:47:06.508828 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 09:47:06.509791 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:47:06.509791 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:47:06.509791 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 09:47:06.509791 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 09:47:06.509791 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 09:47:06.509791 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 09:47:06.509791 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:47:06.509791 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:47:06.509791 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:47:06.509791 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:47:06.509791 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:47:06.509791 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:47:06.509791 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:47:06.509791 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:47:06.509791 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:47:06.509791 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:47:06.509791 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:47:06.509791 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:47:06.510743 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:47:06.510743 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:47:06.510743 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 09:47:06.510743 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 09:47:06.510743 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 09:47:06.510743 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 09:47:06.510743 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 09:47:06.510743 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 09:47:06.510743 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 02 09:47:06.510743 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:47:06.510743 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 437/866] 10.222.0.64 () {56 vars in 1117 bytes} [Tue Aug 2 09:47:06 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:47:06.518152 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86befffa-ca15-46f5-914b-721fccfc669f None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:06.540300 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:06.628996 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86befffa-ca15-46f5-914b-721fccfc669f None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:06.664123 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d635422e-d70b-4434-874c-147e0af2bfaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:06.664703 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d635422e-d70b-4434-874c-147e0af2bfaf None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:06.664987 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d635422e-d70b-4434-874c-147e0af2bfaf None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:06.665263 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d635422e-d70b-4434-874c-147e0af2bfaf None admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:06.666068 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d635422e-d70b-4434-874c-147e0af2bfaf None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:47:06.666375 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d635422e-d70b-4434-874c-147e0af2bfaf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:06.667809 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d635422e-d70b-4434-874c-147e0af2bfaf None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:06.686176 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 430/867] 10.222.0.64 () {56 vars in 1149 bytes} [Tue Aug 2 09:47:06 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:47:06.692477 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b101b82a-832c-4ca5-b85d-7581590565fb None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:06.811053 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b101b82a-832c-4ca5-b85d-7581590565fb None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:06.826682 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:06.865411 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfa7516-928e-48e6-897c-8941ced829e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:06.866433 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfa7516-928e-48e6-897c-8941ced829e5 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:06.866918 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfa7516-928e-48e6-897c-8941ced829e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:06.867392 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfa7516-928e-48e6-897c-8941ced829e5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:06.937714 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfa7516-928e-48e6-897c-8941ced829e5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:06.939138 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfa7516-928e-48e6-897c-8941ced829e5 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:07.104846 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 438/868] 10.222.0.64 () {58 vars in 1282 bytes} [Tue Aug 2 09:47:06 2022] GET /identity/v3/auth/tokens => generated 1262 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 02 09:47:07.114211 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d635422e-d70b-4434-874c-147e0af2bfaf None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:07.257900 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:07.299043 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d635422e-d70b-4434-874c-147e0af2bfaf None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:07.366948 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131ae9f7-b665-46ce-acae-9cf9b172295a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:07.368151 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131ae9f7-b665-46ce-acae-9cf9b172295a None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:07.368714 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131ae9f7-b665-46ce-acae-9cf9b172295a None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:07.369218 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131ae9f7-b665-46ce-acae-9cf9b172295a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:07.372324 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e8e25a-b18f-499b-8a7d-2055ec5d32ec None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:07.372848 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e8e25a-b18f-499b-8a7d-2055ec5d32ec None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:07.373364 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e8e25a-b18f-499b-8a7d-2055ec5d32ec None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:07.376850 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 439/869] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:47:07 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:47:07.382054 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131ae9f7-b665-46ce-acae-9cf9b172295a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:07.383631 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131ae9f7-b665-46ce-acae-9cf9b172295a None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:07.385956 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600b078f-717d-4128-ac73-b99f6fcff393 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:07.386635 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600b078f-717d-4128-ac73-b99f6fcff393 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:07.387159 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600b078f-717d-4128-ac73-b99f6fcff393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:07.400142 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-131ae9f7-b665-46ce-acae-9cf9b172295a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:47:07.400142 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:47:07.400142 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:47:07.400142 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:47:07.400142 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:47:07.400142 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:47:07.400142 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:47:07.400142 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:47:07.400142 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:47:07.400142 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:47:07.400142 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:47:07.400142 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:47:07.400142 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 09:47:07.400142 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 09:47:07.400142 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 09:47:07.401823 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 09:47:07.401823 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:47:07.401823 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:47:07.401823 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:47:07.401823 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:47:07.401823 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:47:07.401823 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:47:07.401823 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:47:07.401823 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:47:07.401823 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:47:07.401823 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:47:07.401823 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:47:07.401823 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:47:07.401823 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:47:07.401823 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:47:07.401823 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:47:07.401823 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:47:07.403618 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 09:47:07.403618 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 09:47:07.403618 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:47:07.403618 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:47:07.403618 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 431/870] 10.222.0.64 () {56 vars in 1108 bytes} [Tue Aug 2 09:47:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:47:07.413932 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131ae9f7-b665-46ce-acae-9cf9b172295a None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:07.416503 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-600b078f-717d-4128-ac73-b99f6fcff393 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:47:07.417702 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-600b078f-717d-4128-ac73-b99f6fcff393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:47:07.428620 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:07.508411 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 440/871] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:47:07 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 02 09:47:07.526490 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600b078f-717d-4128-ac73-b99f6fcff393 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:07.561093 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131ae9f7-b665-46ce-acae-9cf9b172295a None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:07.592618 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:07.600783 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3b00bf-f79f-4730-8d5d-d04579e16f43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:07.601887 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3b00bf-f79f-4730-8d5d-d04579e16f43 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:07.602373 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3b00bf-f79f-4730-8d5d-d04579e16f43 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:07.602847 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3b00bf-f79f-4730-8d5d-d04579e16f43 None admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:07.603745 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3b00bf-f79f-4730-8d5d-d04579e16f43 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:47:07.604322 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3b00bf-f79f-4730-8d5d-d04579e16f43 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:07.606010 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3b00bf-f79f-4730-8d5d-d04579e16f43 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:07.620230 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 432/872] 10.222.0.64 () {56 vars in 1109 bytes} [Tue Aug 2 09:47:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:47:07.626585 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3b00bf-f79f-4730-8d5d-d04579e16f43 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:07.646140 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600b078f-717d-4128-ac73-b99f6fcff393 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:07.693014 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dce43c-5cbb-4ef8-9b62-b2d308a71f95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:07.695052 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dce43c-5cbb-4ef8-9b62-b2d308a71f95 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:07.695052 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dce43c-5cbb-4ef8-9b62-b2d308a71f95 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:07.695052 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dce43c-5cbb-4ef8-9b62-b2d308a71f95 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:07.695052 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dce43c-5cbb-4ef8-9b62-b2d308a71f95 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:07.701959 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dce43c-5cbb-4ef8-9b62-b2d308a71f95 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:07.706380 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-20dce43c-5cbb-4ef8-9b62-b2d308a71f95 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 02 09:47:07.706380 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:47:07.706380 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 09:47:07.706380 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 09:47:07.706380 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 09:47:07.706380 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 09:47:07.706380 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 09:47:07.706380 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 09:47:07.706380 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 02 09:47:07.706380 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:47:07.706380 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 09:47:07.706380 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:47:07.706380 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:47:07.706380 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:47:07.706380 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:47:07.706380 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:47:07.706380 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:47:07.707451 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:47:07.707451 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:47:07.707451 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:47:07.707451 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:47:07.707451 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:47:07.707451 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:47:07.707451 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 09:47:07.707451 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 09:47:07.707451 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 09:47:07.707451 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 09:47:07.707451 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:47:07.707451 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:47:07.707451 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:47:07.707451 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:47:07.707451 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:47:07.707451 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:47:07.708352 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:47:07.708352 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:47:07.708352 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:47:07.708352 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:47:07.708352 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:47:07.708352 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:47:07.708352 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:47:07.708352 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:47:07.708352 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:47:07.708352 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:47:07.708352 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 09:47:07.708352 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 09:47:07.708352 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 02 09:47:07.708352 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:47:07.712379 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 441/873] 10.222.0.64 () {56 vars in 1114 bytes} [Tue Aug 2 09:47:07 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 09:47:07.714090 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ead7d9-a56c-4e51-a964-0f42c3ffe5a6 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:07.714503 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ead7d9-a56c-4e51-a964-0f42c3ffe5a6 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:07.714685 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ead7d9-a56c-4e51-a964-0f42c3ffe5a6 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:07.716129 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 442/874] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:47:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:47:07.719673 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ead7d9-a56c-4e51-a964-0f42c3ffe5a6 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:07.735827 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:07.771966 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3b00bf-f79f-4730-8d5d-d04579e16f43 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:07.810217 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae5bfe6-5d37-4eac-97a8-4c1e67617e03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:07.810858 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae5bfe6-5d37-4eac-97a8-4c1e67617e03 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:07.811272 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae5bfe6-5d37-4eac-97a8-4c1e67617e03 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:07.811426 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae5bfe6-5d37-4eac-97a8-4c1e67617e03 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:07.819216 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae5bfe6-5d37-4eac-97a8-4c1e67617e03 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:07.821319 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae5bfe6-5d37-4eac-97a8-4c1e67617e03 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:07.833522 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-fae5bfe6-5d37-4eac-97a8-4c1e67617e03 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 09:47:07.833522 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:47:07.833522 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:47:07.833522 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:47:07.833522 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:47:07.833522 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:47:07.833522 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:47:07.833522 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:47:07.833522 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:47:07.833522 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:47:07.833522 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:47:07.833522 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:47:07.833522 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 09:47:07.833522 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 09:47:07.833522 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 09:47:07.834552 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 09:47:07.834552 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:47:07.834552 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:47:07.834552 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:47:07.834552 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:47:07.834552 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:47:07.834552 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:47:07.834552 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:47:07.834552 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:47:07.834552 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:47:07.834552 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:47:07.834552 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:47:07.834552 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:47:07.834552 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:47:07.834552 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:47:07.834552 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:47:07.834552 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:47:07.835482 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 09:47:07.835482 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 09:47:07.835482 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 09:47:07.835482 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 09:47:07.835482 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 09:47:07.835482 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 09:47:07.835482 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 09:47:07.835482 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:47:07.835482 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 433/875] 10.222.0.64 () {56 vars in 1111 bytes} [Tue Aug 2 09:47:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:47:07.835994 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824205e3-afa8-4a7b-8dee-621bf7cbc447 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:07.836402 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824205e3-afa8-4a7b-8dee-621bf7cbc447 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:07.836402 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824205e3-afa8-4a7b-8dee-621bf7cbc447 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:07.836998 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ead7d9-a56c-4e51-a964-0f42c3ffe5a6 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:07.856589 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-824205e3-afa8-4a7b-8dee-621bf7cbc447 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:47:07.857784 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-824205e3-afa8-4a7b-8dee-621bf7cbc447 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:47:07.875130 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9a808d-3092-4191-a78f-b46a57017290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:07.875775 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9a808d-3092-4191-a78f-b46a57017290 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:07.876059 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9a808d-3092-4191-a78f-b46a57017290 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:07.876371 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9a808d-3092-4191-a78f-b46a57017290 None admin] PATH_INFO: `/v3/services` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:07.877054 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9a808d-3092-4191-a78f-b46a57017290 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:47:07.877361 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9a808d-3092-4191-a78f-b46a57017290 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:07.878886 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9a808d-3092-4191-a78f-b46a57017290 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:07.883501 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 443/876] 10.222.0.64 () {56 vars in 1114 bytes} [Tue Aug 2 09:47:07 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:47:07.884768 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898e6408-cb5b-437b-a938-4d8195fc4e47 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:07.885057 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898e6408-cb5b-437b-a938-4d8195fc4e47 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:07.885337 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898e6408-cb5b-437b-a938-4d8195fc4e47 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:07.886605 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 444/877] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:47:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:47:07.887313 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898e6408-cb5b-437b-a938-4d8195fc4e47 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:07.911692 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:07.934054 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 434/878] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:47:07 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 09:47:07.934908 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824205e3-afa8-4a7b-8dee-621bf7cbc447 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:07.964744 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:07.999138 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898e6408-cb5b-437b-a938-4d8195fc4e47 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:08.035703 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6805b0b4-ef03-4ce0-abad-ebfc204c2129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:08.036267 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6805b0b4-ef03-4ce0-abad-ebfc204c2129 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:08.036564 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6805b0b4-ef03-4ce0-abad-ebfc204c2129 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:08.036844 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6805b0b4-ef03-4ce0-abad-ebfc204c2129 None admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:08.037609 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6805b0b4-ef03-4ce0-abad-ebfc204c2129 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:47:08.037913 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6805b0b4-ef03-4ce0-abad-ebfc204c2129 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:08.039387 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6805b0b4-ef03-4ce0-abad-ebfc204c2129 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:08.044625 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824205e3-afa8-4a7b-8dee-621bf7cbc447 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:08.052827 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 445/879] 10.222.0.64 () {56 vars in 1148 bytes} [Tue Aug 2 09:47:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:47:08.054076 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a76e0a-b597-4e2d-b88b-4a8c80e14149 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:08.054367 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a76e0a-b597-4e2d-b88b-4a8c80e14149 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:08.054643 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a76e0a-b597-4e2d-b88b-4a8c80e14149 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:08.073474 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-f0a76e0a-b597-4e2d-b88b-4a8c80e14149 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:47:08.074673 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-f0a76e0a-b597-4e2d-b88b-4a8c80e14149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:47:08.088382 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3d1a64-c276-48cc-87d7-6ddbd3797866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:08.089013 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3d1a64-c276-48cc-87d7-6ddbd3797866 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:08.089301 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3d1a64-c276-48cc-87d7-6ddbd3797866 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:08.089586 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3d1a64-c276-48cc-87d7-6ddbd3797866 None admin] PATH_INFO: `/v3/services` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:08.090283 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3d1a64-c276-48cc-87d7-6ddbd3797866 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:47:08.090597 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3d1a64-c276-48cc-87d7-6ddbd3797866 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:08.091978 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3d1a64-c276-48cc-87d7-6ddbd3797866 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:08.097138 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 435/880] 10.222.0.64 () {56 vars in 1114 bytes} [Tue Aug 2 09:47:07 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:47:08.097853 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3d1a64-c276-48cc-87d7-6ddbd3797866 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:08.154132 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 446/881] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:47:08 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 09:47:08.155025 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a76e0a-b597-4e2d-b88b-4a8c80e14149 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:08.165913 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:08.232465 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3d1a64-c276-48cc-87d7-6ddbd3797866 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:08.233038 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:08.269646 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c44f25-7988-44a6-9ae7-2f0d1611ff66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:08.270279 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c44f25-7988-44a6-9ae7-2f0d1611ff66 None admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:08.270567 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c44f25-7988-44a6-9ae7-2f0d1611ff66 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:08.270852 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c44f25-7988-44a6-9ae7-2f0d1611ff66 None admin] PATH_INFO: `/v3/services` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:08.271559 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c44f25-7988-44a6-9ae7-2f0d1611ff66 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:08.272971 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c44f25-7988-44a6-9ae7-2f0d1611ff66 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:08.275369 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a76e0a-b597-4e2d-b88b-4a8c80e14149 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:08.316373 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 436/882] 10.222.0.64 () {60 vars in 1141 bytes} [Tue Aug 2 09:47:08 2022] POST /identity/v3/services => generated 269 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:47:08.316373 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c44f25-7988-44a6-9ae7-2f0d1611ff66 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:08.335417 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68f99b0-09be-4925-b0b7-70f2d0f16b3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:08.336061 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68f99b0-09be-4925-b0b7-70f2d0f16b3f None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:08.336413 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68f99b0-09be-4925-b0b7-70f2d0f16b3f None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:08.336691 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68f99b0-09be-4925-b0b7-70f2d0f16b3f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:08.337519 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68f99b0-09be-4925-b0b7-70f2d0f16b3f None admin] RBAC: Adding query filter params (role.id=5704af09562a457fa35364e8af9481f1, scope.project.id=390bd7c4f91c4465b0c7ab0f9e436c4d, user.id=5338bc8c594e4bc9bc41019226bf8d87) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:47:08.337842 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68f99b0-09be-4925-b0b7-70f2d0f16b3f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:08.339350 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68f99b0-09be-4925-b0b7-70f2d0f16b3f None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:08.346924 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 447/883] 10.222.0.64 () {56 vars in 1374 bytes} [Tue Aug 2 09:47:08 2022] GET /identity/v3/role_assignments?user.id=5338bc8c594e4bc9bc41019226bf8d87&scope.project.id=390bd7c4f91c4465b0c7ab0f9e436c4d&role.id=5704af09562a457fa35364e8af9481f1 => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:47:08.347906 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68f99b0-09be-4925-b0b7-70f2d0f16b3f None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:08.442997 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c44f25-7988-44a6-9ae7-2f0d1611ff66 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:08.447035 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:08.491849 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d60a4e8-48cc-45d5-81b9-7be166cc0253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:08.492759 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d60a4e8-48cc-45d5-81b9-7be166cc0253 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:08.493149 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d60a4e8-48cc-45d5-81b9-7be166cc0253 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:08.493532 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d60a4e8-48cc-45d5-81b9-7be166cc0253 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:08.495823 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d60a4e8-48cc-45d5-81b9-7be166cc0253 None admin] RBAC: Adding query filter params (interface=public, service_id=12ac64b2ece14b02960b2ced28161a02, region_id=RegionOne) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:47:08.495823 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d60a4e8-48cc-45d5-81b9-7be166cc0253 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:08.496732 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d60a4e8-48cc-45d5-81b9-7be166cc0253 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:08.508994 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 437/884] 10.222.0.64 () {56 vars in 1251 bytes} [Tue Aug 2 09:47:08 2022] GET /identity/v3/endpoints?service_id=12ac64b2ece14b02960b2ced28161a02&interface=public®ion_id=RegionOne => generated 194 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:47:08.509926 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d60a4e8-48cc-45d5-81b9-7be166cc0253 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:08.523337 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68f99b0-09be-4925-b0b7-70f2d0f16b3f None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:08.552592 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:08.568732 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30e7e4d-4c04-4e55-9041-7f5e21c90f25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:08.569718 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30e7e4d-4c04-4e55-9041-7f5e21c90f25 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:08.570005 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30e7e4d-4c04-4e55-9041-7f5e21c90f25 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:08.570266 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30e7e4d-4c04-4e55-9041-7f5e21c90f25 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:08.570902 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30e7e4d-4c04-4e55-9041-7f5e21c90f25 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:08.572309 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30e7e4d-4c04-4e55-9041-7f5e21c90f25 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:08.581973 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-b30e7e4d-4c04-4e55-9041-7f5e21c90f25 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 02 09:47:08.581973 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:47:08.581973 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 09:47:08.581973 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 09:47:08.581973 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 09:47:08.581973 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 09:47:08.581973 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 09:47:08.581973 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 09:47:08.581973 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 02 09:47:08.581973 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:47:08.581973 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 09:47:08.581973 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:47:08.581973 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:47:08.581973 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:47:08.581973 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:47:08.581973 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:47:08.581973 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:47:08.583077 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:47:08.583077 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:47:08.583077 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:47:08.583077 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:47:08.583077 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:47:08.583077 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:47:08.583077 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 09:47:08.583077 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 09:47:08.583077 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 09:47:08.583077 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 09:47:08.583077 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:47:08.583077 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:47:08.583077 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:47:08.583077 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:47:08.583077 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:47:08.583077 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:47:08.584024 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:47:08.584024 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:47:08.584024 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:47:08.584024 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:47:08.584024 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:47:08.584024 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:47:08.584024 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:47:08.584024 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:47:08.584024 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:47:08.584024 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:47:08.584024 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 09:47:08.584024 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 09:47:08.584024 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 02 09:47:08.584024 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:47:08.584024 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 448/885] 10.222.0.64 () {56 vars in 1126 bytes} [Tue Aug 2 09:47:08 2022] GET /identity/v3/services/object-store => generated 93 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:47:08.590750 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30e7e4d-4c04-4e55-9041-7f5e21c90f25 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:08.675480 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d60a4e8-48cc-45d5-81b9-7be166cc0253 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:08.717556 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31914b61-48ff-4dee-bb2d-13fb31714ed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:08.718841 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31914b61-48ff-4dee-bb2d-13fb31714ed7 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:08.719117 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31914b61-48ff-4dee-bb2d-13fb31714ed7 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:08.719650 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31914b61-48ff-4dee-bb2d-13fb31714ed7 None admin] PATH_INFO: `/v3/services` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:08.720869 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31914b61-48ff-4dee-bb2d-13fb31714ed7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:08.722816 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31914b61-48ff-4dee-bb2d-13fb31714ed7 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:08.726681 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30e7e4d-4c04-4e55-9041-7f5e21c90f25 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:08.731633 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 438/886] 10.222.0.64 () {56 vars in 1087 bytes} [Tue Aug 2 09:47:08 2022] GET /identity/v3/services => generated 1268 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 09:47:08.767551 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddf281e-f864-4b89-8d6e-6553bc387ebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:08.768177 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddf281e-f864-4b89-8d6e-6553bc387ebc None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:08.768507 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddf281e-f864-4b89-8d6e-6553bc387ebc None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:08.768774 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddf281e-f864-4b89-8d6e-6553bc387ebc None admin] PATH_INFO: `/v3/services` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:08.769445 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddf281e-f864-4b89-8d6e-6553bc387ebc None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:47:08.769750 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddf281e-f864-4b89-8d6e-6553bc387ebc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:08.771134 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddf281e-f864-4b89-8d6e-6553bc387ebc None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:08.777073 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 449/887] 10.222.0.64 () {56 vars in 1122 bytes} [Tue Aug 2 09:47:08 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:47:08.783354 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31914b61-48ff-4dee-bb2d-13fb31714ed7 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:08.820759 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:08.906558 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31914b61-48ff-4dee-bb2d-13fb31714ed7 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:08.948376 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f110036-3384-4ff0-80dd-6de474255e58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:08.949000 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f110036-3384-4ff0-80dd-6de474255e58 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:08.949284 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f110036-3384-4ff0-80dd-6de474255e58 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:08.949556 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f110036-3384-4ff0-80dd-6de474255e58 None admin] PATH_INFO: `/v3/services` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:08.950365 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f110036-3384-4ff0-80dd-6de474255e58 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:47:08.950671 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f110036-3384-4ff0-80dd-6de474255e58 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:08.952070 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f110036-3384-4ff0-80dd-6de474255e58 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:08.958336 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 439/888] 10.222.0.64 () {56 vars in 1122 bytes} [Tue Aug 2 09:47:08 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:47:10.460330 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff63d37-ae08-46a0-9370-1b335e9bfc7b None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:10.461217 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff63d37-ae08-46a0-9370-1b335e9bfc7b None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:10.461645 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff63d37-ae08-46a0-9370-1b335e9bfc7b None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:10.463396 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 450/889] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:47:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:47:10.468960 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb6e736-e984-4a5b-bc32-fd9981ec6856 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:10.469829 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb6e736-e984-4a5b-bc32-fd9981ec6856 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:10.470334 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb6e736-e984-4a5b-bc32-fd9981ec6856 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:10.489399 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-abb6e736-e984-4a5b-bc32-fd9981ec6856 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:47:10.490946 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-abb6e736-e984-4a5b-bc32-fd9981ec6856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:47:10.560995 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 440/890] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:47:10 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 09:47:10.583795 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff63d37-ae08-46a0-9370-1b335e9bfc7b None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:10.601951 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c56eb6-139c-42b5-b375-807e69e8136c None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:10.602260 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c56eb6-139c-42b5-b375-807e69e8136c None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:10.602518 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c56eb6-139c-42b5-b375-807e69e8136c None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:10.604924 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 441/891] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:47:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 09:47:10.611417 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a738e4b-fa9f-4af8-be05-fc74d4fef321 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:10.611703 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a738e4b-fa9f-4af8-be05-fc74d4fef321 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:10.611982 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a738e4b-fa9f-4af8-be05-fc74d4fef321 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:10.634947 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-6a738e4b-fa9f-4af8-be05-fc74d4fef321 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:47:10.636640 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-6a738e4b-fa9f-4af8-be05-fc74d4fef321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:47:10.644676 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:10.667145 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:10.707245 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff63d37-ae08-46a0-9370-1b335e9bfc7b None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:10.734391 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 442/892] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:47:10 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 02 09:47:10.746335 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baf0efd-195d-4ac0-a92b-d720a146cc51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:10.747299 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baf0efd-195d-4ac0-a92b-d720a146cc51 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:10.747799 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baf0efd-195d-4ac0-a92b-d720a146cc51 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:10.748363 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baf0efd-195d-4ac0-a92b-d720a146cc51 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:10.749229 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baf0efd-195d-4ac0-a92b-d720a146cc51 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:10.751340 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baf0efd-195d-4ac0-a92b-d720a146cc51 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:10.758188 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a738e4b-fa9f-4af8-be05-fc74d4fef321 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:10.765063 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-0baf0efd-195d-4ac0-a92b-d720a146cc51 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 02 09:47:10.765063 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:47:10.765063 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 09:47:10.765063 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 09:47:10.765063 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 09:47:10.765063 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 09:47:10.765063 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 09:47:10.765063 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 09:47:10.765063 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 02 09:47:10.765063 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:47:10.765063 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 09:47:10.765063 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:47:10.765063 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:47:10.765063 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:47:10.765063 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:47:10.765063 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:47:10.766068 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:47:10.766068 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:47:10.766068 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:47:10.766068 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:47:10.766068 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:47:10.766068 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:47:10.766068 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:47:10.766068 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 09:47:10.766068 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 09:47:10.766068 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 09:47:10.766068 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 09:47:10.766068 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:47:10.766068 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:47:10.766068 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:47:10.766068 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:47:10.766068 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:47:10.766922 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:47:10.766922 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:47:10.766922 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:47:10.766922 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:47:10.766922 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:47:10.766922 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:47:10.766922 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:47:10.766922 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:47:10.766922 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:47:10.766922 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:47:10.766922 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:47:10.766922 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 09:47:10.766922 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 09:47:10.766922 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 02 09:47:10.766922 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:47:10.769851 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 451/893] 10.222.0.64 () {56 vars in 1132 bytes} [Tue Aug 2 09:47:10 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:47:10.775573 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baf0efd-195d-4ac0-a92b-d720a146cc51 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:10.875129 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a738e4b-fa9f-4af8-be05-fc74d4fef321 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:10.881954 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baf0efd-195d-4ac0-a92b-d720a146cc51 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:10.899351 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:10.916890 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858e80a3-b5b4-4456-b873-df1b317dd19b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:10.917534 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858e80a3-b5b4-4456-b873-df1b317dd19b None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:10.917819 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858e80a3-b5b4-4456-b873-df1b317dd19b None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:10.918094 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858e80a3-b5b4-4456-b873-df1b317dd19b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:10.923124 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8414ee71-c3ea-4093-957e-0d4c97839c54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:10.923936 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8414ee71-c3ea-4093-957e-0d4c97839c54 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:10.924613 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8414ee71-c3ea-4093-957e-0d4c97839c54 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:10.925082 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8414ee71-c3ea-4093-957e-0d4c97839c54 None admin] PATH_INFO: `/v3/services` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:10.926069 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8414ee71-c3ea-4093-957e-0d4c97839c54 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:47:10.926612 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8414ee71-c3ea-4093-957e-0d4c97839c54 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:10.928179 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8414ee71-c3ea-4093-957e-0d4c97839c54 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:10.929094 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858e80a3-b5b4-4456-b873-df1b317dd19b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 09:47:10.934163 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-858e80a3-b5b4-4456-b873-df1b317dd19b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 02 09:47:10.934163 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:47:10.934163 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:47:10.934163 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:47:10.934163 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:47:10.934163 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:47:10.934163 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:47:10.934163 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:47:10.934163 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:47:10.934163 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:47:10.934163 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:47:10.934163 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:47:10.934163 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 09:47:10.934163 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 09:47:10.934163 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 09:47:10.934163 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 09:47:10.935125 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 09:47:10.935125 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 09:47:10.935125 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 09:47:10.935125 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 09:47:10.935125 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:47:10.935125 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:47:10.935125 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:47:10.935125 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:47:10.935125 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:47:10.935125 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:47:10.935125 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:47:10.935125 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:47:10.935125 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:47:10.935125 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:47:10.935125 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:47:10.935125 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:47:10.935984 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:47:10.935984 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:47:10.935984 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:47:10.935984 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:47:10.935984 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 09:47:10.935984 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 09:47:10.935984 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 09:47:10.935984 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 09:47:10.935984 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 09:47:10.935984 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 09:47:10.935984 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 02 09:47:10.935984 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:47:10.937824 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 452/894] 10.222.0.64 () {56 vars in 1126 bytes} [Tue Aug 2 09:47:10 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:47:10.939100 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec4c3b3-9f15-4602-8fa5-e2065671d3b4 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:10.939395 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec4c3b3-9f15-4602-8fa5-e2065671d3b4 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:10.939671 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec4c3b3-9f15-4602-8fa5-e2065671d3b4 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:10.940129 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 443/895] 10.222.0.64 () {56 vars in 1120 bytes} [Tue Aug 2 09:47:10 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 09:47:10.940845 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858e80a3-b5b4-4456-b873-df1b317dd19b None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:10.940953 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 453/896] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:47:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:47:10.944645 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec4c3b3-9f15-4602-8fa5-e2065671d3b4 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:10.968547 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:11.049391 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec4c3b3-9f15-4602-8fa5-e2065671d3b4 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:11.059329 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858e80a3-b5b4-4456-b873-df1b317dd19b None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:11.087099 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98569b5e-38ea-4a71-9a3f-4ad5579aa0aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:11.087717 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98569b5e-38ea-4a71-9a3f-4ad5579aa0aa None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:11.088003 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98569b5e-38ea-4a71-9a3f-4ad5579aa0aa None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:11.088311 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98569b5e-38ea-4a71-9a3f-4ad5579aa0aa None admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:11.089031 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98569b5e-38ea-4a71-9a3f-4ad5579aa0aa None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:47:11.089338 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98569b5e-38ea-4a71-9a3f-4ad5579aa0aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:11.090707 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98569b5e-38ea-4a71-9a3f-4ad5579aa0aa None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:11.095195 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53595216-ab49-46ed-9977-3a5bc3a74323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:11.095806 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53595216-ab49-46ed-9977-3a5bc3a74323 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:11.096546 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53595216-ab49-46ed-9977-3a5bc3a74323 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:11.096830 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53595216-ab49-46ed-9977-3a5bc3a74323 None admin] PATH_INFO: `/v3/services` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:11.097510 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53595216-ab49-46ed-9977-3a5bc3a74323 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:47:11.097814 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53595216-ab49-46ed-9977-3a5bc3a74323 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:11.104108 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 454/897] 10.222.0.64 () {56 vars in 1115 bytes} [Tue Aug 2 09:47:10 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:47:11.104294 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53595216-ab49-46ed-9977-3a5bc3a74323 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:11.104388 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1279a1-0174-443f-adac-c854a365785f None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:11.104388 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1279a1-0174-443f-adac-c854a365785f None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:11.104388 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1279a1-0174-443f-adac-c854a365785f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:11.113455 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 444/898] 10.222.0.64 () {56 vars in 1126 bytes} [Tue Aug 2 09:47:10 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:47:11.113455 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53595216-ab49-46ed-9977-3a5bc3a74323 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:11.125899 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-9b1279a1-0174-443f-adac-c854a365785f None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:47:11.127127 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-9b1279a1-0174-443f-adac-c854a365785f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:47:11.166538 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:11.200555 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:11.207751 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 455/899] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:47:11 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 09:47:11.208920 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1279a1-0174-443f-adac-c854a365785f None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:11.235124 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53595216-ab49-46ed-9977-3a5bc3a74323 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:11.271923 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41b63cf-e760-40f3-bf6b-1c9575ad5d7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:11.272603 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41b63cf-e760-40f3-bf6b-1c9575ad5d7e None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:11.272889 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41b63cf-e760-40f3-bf6b-1c9575ad5d7e None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:11.273451 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41b63cf-e760-40f3-bf6b-1c9575ad5d7e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:11.284967 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41b63cf-e760-40f3-bf6b-1c9575ad5d7e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:11.286409 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41b63cf-e760-40f3-bf6b-1c9575ad5d7e None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:11.303529 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-c41b63cf-e760-40f3-bf6b-1c9575ad5d7e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:47:11.303529 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:47:11.303529 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:47:11.303529 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:47:11.303529 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:47:11.303529 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:47:11.303529 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:47:11.303529 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:47:11.303529 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:47:11.303529 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:47:11.303529 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:47:11.303529 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:47:11.303529 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 09:47:11.303529 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 09:47:11.303529 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 09:47:11.304601 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 09:47:11.304601 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:47:11.304601 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:47:11.304601 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:47:11.304601 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:47:11.304601 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:47:11.304601 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:47:11.304601 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:47:11.304601 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:47:11.304601 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:47:11.304601 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:47:11.304601 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:47:11.304601 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:47:11.304601 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:47:11.304601 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:47:11.304601 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:47:11.304601 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:47:11.305654 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 09:47:11.305654 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 09:47:11.305654 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:47:11.305654 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:47:11.305654 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 445/900] 10.222.0.64 () {56 vars in 1108 bytes} [Tue Aug 2 09:47:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:47:11.305654 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41b63cf-e760-40f3-bf6b-1c9575ad5d7e None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:11.325261 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1279a1-0174-443f-adac-c854a365785f None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:11.362514 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35998a9e-c7b3-4fd6-acfa-030c4006e5d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:11.363142 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35998a9e-c7b3-4fd6-acfa-030c4006e5d9 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:11.363433 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35998a9e-c7b3-4fd6-acfa-030c4006e5d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:11.363706 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35998a9e-c7b3-4fd6-acfa-030c4006e5d9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:11.364486 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35998a9e-c7b3-4fd6-acfa-030c4006e5d9 None admin] RBAC: Adding query filter params (interface=public, service_id=27223137e7944b889ea30320e0f7cbb4, region_id=RegionOne) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:47:11.364799 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35998a9e-c7b3-4fd6-acfa-030c4006e5d9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:11.366302 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35998a9e-c7b3-4fd6-acfa-030c4006e5d9 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:11.376662 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 456/901] 10.222.0.64 () {56 vars in 1251 bytes} [Tue Aug 2 09:47:11 2022] GET /identity/v3/endpoints?service_id=27223137e7944b889ea30320e0f7cbb4&interface=public®ion_id=RegionOne => generated 194 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:47:11.377507 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35998a9e-c7b3-4fd6-acfa-030c4006e5d9 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:11.412854 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:11.416635 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41b63cf-e760-40f3-bf6b-1c9575ad5d7e None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:11.449037 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:11.456362 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99afed37-97d7-4b87-acdd-2307d390b922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:11.457008 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99afed37-97d7-4b87-acdd-2307d390b922 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:11.457293 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99afed37-97d7-4b87-acdd-2307d390b922 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:11.457570 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99afed37-97d7-4b87-acdd-2307d390b922 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:11.458204 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99afed37-97d7-4b87-acdd-2307d390b922 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:11.459581 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99afed37-97d7-4b87-acdd-2307d390b922 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:11.470538 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-99afed37-97d7-4b87-acdd-2307d390b922 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 02 09:47:11.470538 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:47:11.470538 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 09:47:11.470538 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 09:47:11.470538 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 09:47:11.470538 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 09:47:11.470538 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 09:47:11.470538 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 09:47:11.470538 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 02 09:47:11.470538 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:47:11.470538 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 09:47:11.470538 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:47:11.470538 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:47:11.470538 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:47:11.470538 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:47:11.470538 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:47:11.470538 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:47:11.472068 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:47:11.472068 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:47:11.472068 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:47:11.472068 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:47:11.472068 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:47:11.472068 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:47:11.472068 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 09:47:11.472068 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 09:47:11.472068 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 09:47:11.472068 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 09:47:11.472068 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:47:11.472068 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:47:11.472068 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:47:11.472068 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:47:11.472068 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:47:11.472068 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:47:11.472975 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:47:11.472975 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:47:11.472975 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:47:11.472975 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:47:11.472975 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:47:11.472975 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:47:11.472975 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:47:11.472975 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:47:11.472975 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:47:11.472975 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:47:11.472975 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 09:47:11.472975 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 09:47:11.472975 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 02 09:47:11.472975 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:47:11.472975 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 446/902] 10.222.0.64 () {56 vars in 1114 bytes} [Tue Aug 2 09:47:11 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:47:11.473802 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99afed37-97d7-4b87-acdd-2307d390b922 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:11.493897 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35998a9e-c7b3-4fd6-acfa-030c4006e5d9 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:11.531383 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b834c14-4ff0-419a-aeb7-28a1a2ab5967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:11.532144 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b834c14-4ff0-419a-aeb7-28a1a2ab5967 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:11.532430 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b834c14-4ff0-419a-aeb7-28a1a2ab5967 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:11.532706 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b834c14-4ff0-419a-aeb7-28a1a2ab5967 None admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:11.533397 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b834c14-4ff0-419a-aeb7-28a1a2ab5967 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:47:11.533704 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b834c14-4ff0-419a-aeb7-28a1a2ab5967 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:11.535306 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b834c14-4ff0-419a-aeb7-28a1a2ab5967 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:11.552013 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 457/903] 10.222.0.64 () {56 vars in 1109 bytes} [Tue Aug 2 09:47:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:47:11.552013 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b834c14-4ff0-419a-aeb7-28a1a2ab5967 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:11.585440 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99afed37-97d7-4b87-acdd-2307d390b922 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:11.621970 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587e95d0-9a4f-4ca2-9711-3e55e5214c83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:11.622588 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587e95d0-9a4f-4ca2-9711-3e55e5214c83 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:11.622876 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587e95d0-9a4f-4ca2-9711-3e55e5214c83 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:11.623155 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587e95d0-9a4f-4ca2-9711-3e55e5214c83 None admin] PATH_INFO: `/v3/services` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:11.623824 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587e95d0-9a4f-4ca2-9711-3e55e5214c83 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:11.625806 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587e95d0-9a4f-4ca2-9711-3e55e5214c83 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:11.632613 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 447/904] 10.222.0.64 () {56 vars in 1087 bytes} [Tue Aug 2 09:47:11 2022] GET /identity/v3/services => generated 1268 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:47:11.633319 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587e95d0-9a4f-4ca2-9711-3e55e5214c83 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:11.671998 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:11.680107 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b834c14-4ff0-419a-aeb7-28a1a2ab5967 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:11.712278 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:11.722171 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d258bb-2a94-4a8c-a253-1ce6c21341d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:11.722815 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d258bb-2a94-4a8c-a253-1ce6c21341d0 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:11.723105 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d258bb-2a94-4a8c-a253-1ce6c21341d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:11.723395 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d258bb-2a94-4a8c-a253-1ce6c21341d0 None admin] PATH_INFO: `/v3/services` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:11.724424 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d258bb-2a94-4a8c-a253-1ce6c21341d0 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:47:11.724734 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d258bb-2a94-4a8c-a253-1ce6c21341d0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:11.726217 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d258bb-2a94-4a8c-a253-1ce6c21341d0 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:11.736308 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 458/905] 10.222.0.64 () {56 vars in 1114 bytes} [Tue Aug 2 09:47:11 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:47:11.736308 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b61b05-9828-4256-85fa-b3cb6d9a156b None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:11.736308 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b61b05-9828-4256-85fa-b3cb6d9a156b None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:11.736308 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b61b05-9828-4256-85fa-b3cb6d9a156b None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:11.736308 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 459/906] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:47:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:47:11.740614 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320aa4cb-4b65-4603-bdce-49654c930725 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:11.740906 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320aa4cb-4b65-4603-bdce-49654c930725 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:11.741188 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320aa4cb-4b65-4603-bdce-49654c930725 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:11.760294 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-320aa4cb-4b65-4603-bdce-49654c930725 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:47:11.761551 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-320aa4cb-4b65-4603-bdce-49654c930725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:47:11.766630 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587e95d0-9a4f-4ca2-9711-3e55e5214c83 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:11.807719 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48d930f-ffd6-4b37-b967-386a7c812fd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:11.808465 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48d930f-ffd6-4b37-b967-386a7c812fd6 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:11.808749 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48d930f-ffd6-4b37-b967-386a7c812fd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:11.809025 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48d930f-ffd6-4b37-b967-386a7c812fd6 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:11.818182 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48d930f-ffd6-4b37-b967-386a7c812fd6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:11.819758 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48d930f-ffd6-4b37-b967-386a7c812fd6 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:11.832073 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-a48d930f-ffd6-4b37-b967-386a7c812fd6 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 02 09:47:11.832073 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:47:11.832073 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:47:11.832073 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:47:11.832073 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:47:11.832073 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:47:11.832073 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:47:11.832073 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:47:11.832073 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:47:11.832073 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:47:11.832073 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:47:11.832073 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:47:11.832073 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 09:47:11.832073 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 09:47:11.832073 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 09:47:11.832073 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 09:47:11.833303 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:47:11.833303 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:47:11.833303 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 09:47:11.833303 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 09:47:11.833303 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 09:47:11.833303 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 09:47:11.833303 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:47:11.833303 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:47:11.833303 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:47:11.833303 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:47:11.833303 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:47:11.833303 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:47:11.833303 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:47:11.833303 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:47:11.833303 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:47:11.833303 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:47:11.833303 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:47:11.833303 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:47:11.834328 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:47:11.834328 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:47:11.834328 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 09:47:11.834328 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 09:47:11.834328 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 09:47:11.834328 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 09:47:11.834328 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 09:47:11.834328 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 09:47:11.834328 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 02 09:47:11.834328 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:47:11.834328 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 448/907] 10.222.0.64 () {56 vars in 1117 bytes} [Tue Aug 2 09:47:11 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:47:11.834328 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48d930f-ffd6-4b37-b967-386a7c812fd6 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:11.846638 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 460/908] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:47:11 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 09:47:11.847823 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320aa4cb-4b65-4603-bdce-49654c930725 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:11.951343 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:11.959692 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48d930f-ffd6-4b37-b967-386a7c812fd6 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:11.965542 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320aa4cb-4b65-4603-bdce-49654c930725 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:11.981464 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:11.998213 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a43ef7-b3d9-434d-a916-0dc16b940ef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:11.998851 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a43ef7-b3d9-434d-a916-0dc16b940ef1 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:11.999171 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a43ef7-b3d9-434d-a916-0dc16b940ef1 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:11.999983 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a43ef7-b3d9-434d-a916-0dc16b940ef1 None admin] PATH_INFO: `/v3/services` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:12.000768 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a43ef7-b3d9-434d-a916-0dc16b940ef1 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:47:12.001078 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a43ef7-b3d9-434d-a916-0dc16b940ef1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:12.002498 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a43ef7-b3d9-434d-a916-0dc16b940ef1 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:12.005937 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a8eb1c-d6dd-40ac-a505-29c74f100692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:12.006793 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a8eb1c-d6dd-40ac-a505-29c74f100692 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:12.007222 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a8eb1c-d6dd-40ac-a505-29c74f100692 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:12.007637 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a8eb1c-d6dd-40ac-a505-29c74f100692 None admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:12.008296 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 449/909] 10.222.0.64 () {56 vars in 1114 bytes} [Tue Aug 2 09:47:11 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:47:12.008805 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a8eb1c-d6dd-40ac-a505-29c74f100692 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:47:12.009125 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a43ef7-b3d9-434d-a916-0dc16b940ef1 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:12.009404 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a8eb1c-d6dd-40ac-a505-29c74f100692 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:12.011056 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a8eb1c-d6dd-40ac-a505-29c74f100692 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:12.030936 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 461/910] 10.222.0.64 () {56 vars in 1149 bytes} [Tue Aug 2 09:47:11 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:47:12.031757 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a8eb1c-d6dd-40ac-a505-29c74f100692 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:12.125665 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a43ef7-b3d9-434d-a916-0dc16b940ef1 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:12.140193 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a8eb1c-d6dd-40ac-a505-29c74f100692 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:12.159923 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10267753-9bef-4e43-b3f0-fe198e06098f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:12.160590 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10267753-9bef-4e43-b3f0-fe198e06098f None admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:12.160872 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10267753-9bef-4e43-b3f0-fe198e06098f None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:12.161147 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10267753-9bef-4e43-b3f0-fe198e06098f None admin] PATH_INFO: `/v3/services` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:12.161869 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10267753-9bef-4e43-b3f0-fe198e06098f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:12.163247 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10267753-9bef-4e43-b3f0-fe198e06098f None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:12.178903 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553735cf-b2bf-4cfa-877c-aa5ea9727732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:12.179772 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553735cf-b2bf-4cfa-877c-aa5ea9727732 None admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:12.179885 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 450/911] 10.222.0.64 () {60 vars in 1141 bytes} [Tue Aug 2 09:47:12 2022] POST /identity/v3/services => generated 241 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:47:12.180312 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553735cf-b2bf-4cfa-877c-aa5ea9727732 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:12.180652 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10267753-9bef-4e43-b3f0-fe198e06098f None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:12.180985 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553735cf-b2bf-4cfa-877c-aa5ea9727732 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:12.181847 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553735cf-b2bf-4cfa-877c-aa5ea9727732 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:12.183566 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553735cf-b2bf-4cfa-877c-aa5ea9727732 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:12.224752 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 462/912] 10.222.0.64 () {60 vars in 1144 bytes} [Tue Aug 2 09:47:12 2022] POST /identity/v3/endpoints => generated 346 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:47:12.260675 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:12.327924 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10267753-9bef-4e43-b3f0-fe198e06098f None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:12.379203 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb7cec9-c227-4a1f-a59e-efd56a4cae5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:12.379767 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb7cec9-c227-4a1f-a59e-efd56a4cae5c None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:12.380056 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb7cec9-c227-4a1f-a59e-efd56a4cae5c None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:12.380370 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb7cec9-c227-4a1f-a59e-efd56a4cae5c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:12.458880 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb7cec9-c227-4a1f-a59e-efd56a4cae5c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:12.460385 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb7cec9-c227-4a1f-a59e-efd56a4cae5c None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:12.495812 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:12.568365 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 451/913] 10.222.0.64 () {58 vars in 1282 bytes} [Tue Aug 2 09:47:12 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 02 09:47:12.573861 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553735cf-b2bf-4cfa-877c-aa5ea9727732 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:12.583590 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:12.684639 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553735cf-b2bf-4cfa-877c-aa5ea9727732 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:12.722634 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7fb556-4bb7-4667-81e2-3712364bf73e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:12.723454 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7fb556-4bb7-4667-81e2-3712364bf73e None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:12.723845 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7fb556-4bb7-4667-81e2-3712364bf73e None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:12.724499 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7fb556-4bb7-4667-81e2-3712364bf73e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:12.737020 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7fb556-4bb7-4667-81e2-3712364bf73e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:12.738690 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7fb556-4bb7-4667-81e2-3712364bf73e None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:12.754463 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-6b7fb556-4bb7-4667-81e2-3712364bf73e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:47:12.754463 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:47:12.754463 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:47:12.754463 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:47:12.754463 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:47:12.754463 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:47:12.754463 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:47:12.754463 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:47:12.754463 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:47:12.754463 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:47:12.754463 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:47:12.754463 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:47:12.754463 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 09:47:12.754463 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 09:47:12.754463 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 09:47:12.755588 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 09:47:12.755588 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:47:12.755588 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:47:12.755588 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:47:12.755588 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:47:12.755588 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:47:12.755588 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:47:12.755588 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:47:12.755588 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:47:12.755588 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:47:12.755588 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:47:12.755588 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:47:12.755588 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:47:12.755588 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:47:12.755588 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:47:12.755588 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:47:12.755588 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:47:12.756722 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 09:47:12.756722 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 09:47:12.756722 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:47:12.756722 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:47:12.758277 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 463/914] 10.222.0.64 () {56 vars in 1108 bytes} [Tue Aug 2 09:47:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:47:12.769577 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb7cec9-c227-4a1f-a59e-efd56a4cae5c None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:12.877648 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb7cec9-c227-4a1f-a59e-efd56a4cae5c None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:12.912853 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3635df-aae6-48d1-a831-8cebc7c9fea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:12.913474 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3635df-aae6-48d1-a831-8cebc7c9fea7 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:12.913758 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3635df-aae6-48d1-a831-8cebc7c9fea7 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:12.914036 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3635df-aae6-48d1-a831-8cebc7c9fea7 None admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:12.914737 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3635df-aae6-48d1-a831-8cebc7c9fea7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:47:12.915045 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3635df-aae6-48d1-a831-8cebc7c9fea7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:12.916444 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3635df-aae6-48d1-a831-8cebc7c9fea7 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:12.918831 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:12.929576 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 452/915] 10.222.0.64 () {56 vars in 1109 bytes} [Tue Aug 2 09:47:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:47:12.935620 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7fb556-4bb7-4667-81e2-3712364bf73e None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:13.020611 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:13.044118 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7fb556-4bb7-4667-81e2-3712364bf73e None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:13.079993 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef64e18f-cf35-4aeb-a0a9-b8422d384bef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:13.080745 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef64e18f-cf35-4aeb-a0a9-b8422d384bef None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:13.081034 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef64e18f-cf35-4aeb-a0a9-b8422d384bef None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:13.081312 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef64e18f-cf35-4aeb-a0a9-b8422d384bef None admin] PATH_INFO: `/v3/projects/service` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:13.088422 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef64e18f-cf35-4aeb-a0a9-b8422d384bef None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:13.089941 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef64e18f-cf35-4aeb-a0a9-b8422d384bef None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:13.102837 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-ef64e18f-cf35-4aeb-a0a9-b8422d384bef None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 09:47:13.102837 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:47:13.102837 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:47:13.102837 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:47:13.102837 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:47:13.102837 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:47:13.102837 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:47:13.102837 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:47:13.102837 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:47:13.102837 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:47:13.102837 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:47:13.102837 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:47:13.102837 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 09:47:13.102837 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 09:47:13.102837 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 09:47:13.103785 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 09:47:13.103785 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:47:13.103785 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:47:13.103785 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:47:13.103785 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:47:13.103785 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:47:13.103785 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:47:13.103785 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:47:13.103785 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:47:13.103785 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:47:13.103785 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:47:13.103785 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:47:13.103785 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:47:13.103785 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:47:13.103785 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:47:13.103785 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:47:13.103785 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:47:13.104725 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 09:47:13.104725 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 09:47:13.104725 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 09:47:13.104725 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 09:47:13.104725 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 09:47:13.104725 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 09:47:13.104725 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 09:47:13.104725 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:47:13.108349 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 464/916] 10.222.0.64 () {56 vars in 1111 bytes} [Tue Aug 2 09:47:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:47:13.110148 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3635df-aae6-48d1-a831-8cebc7c9fea7 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:13.218550 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3635df-aae6-48d1-a831-8cebc7c9fea7 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:13.255950 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8a5d95-875e-4c31-8a36-2f10a3df6d30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:13.256602 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8a5d95-875e-4c31-8a36-2f10a3df6d30 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:13.256891 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8a5d95-875e-4c31-8a36-2f10a3df6d30 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:13.257168 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8a5d95-875e-4c31-8a36-2f10a3df6d30 None admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:13.257933 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8a5d95-875e-4c31-8a36-2f10a3df6d30 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:47:13.258244 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8a5d95-875e-4c31-8a36-2f10a3df6d30 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:13.259689 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8a5d95-875e-4c31-8a36-2f10a3df6d30 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:13.273519 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 453/917] 10.222.0.64 () {56 vars in 1148 bytes} [Tue Aug 2 09:47:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:47:13.279202 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef64e18f-cf35-4aeb-a0a9-b8422d384bef None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:13.386795 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef64e18f-cf35-4aeb-a0a9-b8422d384bef None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:13.401517 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:13.424736 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a98c479-cd37-4d6e-b527-9af82a1dbe71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:13.425377 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a98c479-cd37-4d6e-b527-9af82a1dbe71 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:13.425665 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a98c479-cd37-4d6e-b527-9af82a1dbe71 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:13.425944 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a98c479-cd37-4d6e-b527-9af82a1dbe71 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:13.426767 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a98c479-cd37-4d6e-b527-9af82a1dbe71 None admin] RBAC: Adding query filter params (role.id=55b4c595b54f40c6b03ed7161e5258bf, scope.project.id=390bd7c4f91c4465b0c7ab0f9e436c4d, user.id=5338bc8c594e4bc9bc41019226bf8d87) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:47:13.427130 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a98c479-cd37-4d6e-b527-9af82a1dbe71 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:13.428660 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a98c479-cd37-4d6e-b527-9af82a1dbe71 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:13.435804 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 465/918] 10.222.0.64 () {56 vars in 1374 bytes} [Tue Aug 2 09:47:13 2022] GET /identity/v3/role_assignments?user.id=5338bc8c594e4bc9bc41019226bf8d87&scope.project.id=390bd7c4f91c4465b0c7ab0f9e436c4d&role.id=55b4c595b54f40c6b03ed7161e5258bf => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:47:13.701250 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc4af25-df8f-4209-8686-c9e9fd1379a3 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:13.701808 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc4af25-df8f-4209-8686-c9e9fd1379a3 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:13.701808 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc4af25-df8f-4209-8686-c9e9fd1379a3 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:13.704139 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 454/919] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:47:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:47:13.709653 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b638ab1d-1f25-4638-a55f-83a8be14e5e3 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:13.710113 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b638ab1d-1f25-4638-a55f-83a8be14e5e3 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:13.710194 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b638ab1d-1f25-4638-a55f-83a8be14e5e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:13.733132 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-b638ab1d-1f25-4638-a55f-83a8be14e5e3 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:47:13.734949 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-b638ab1d-1f25-4638-a55f-83a8be14e5e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:47:13.809085 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 466/920] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:47:13 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 02 09:47:13.833798 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc4af25-df8f-4209-8686-c9e9fd1379a3 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:13.902000 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:13.950040 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc4af25-df8f-4209-8686-c9e9fd1379a3 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:13.986632 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af7752c-cec5-4d10-84f6-c4c0dbec4b2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:13.987196 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af7752c-cec5-4d10-84f6-c4c0dbec4b2b None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:13.987480 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af7752c-cec5-4d10-84f6-c4c0dbec4b2b None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:13.987759 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af7752c-cec5-4d10-84f6-c4c0dbec4b2b None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:13.988421 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af7752c-cec5-4d10-84f6-c4c0dbec4b2b None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:13.989859 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af7752c-cec5-4d10-84f6-c4c0dbec4b2b None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:13.999224 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-9af7752c-cec5-4d10-84f6-c4c0dbec4b2b None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 02 09:47:13.999224 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:47:13.999224 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 09:47:13.999224 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 09:47:13.999224 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 09:47:13.999224 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 09:47:13.999224 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 09:47:13.999224 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 09:47:13.999224 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 02 09:47:13.999224 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:47:13.999224 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 09:47:13.999224 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:47:13.999224 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:47:13.999224 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:47:13.999224 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:47:13.999224 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:47:14.000342 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:47:14.000342 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:47:14.000342 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:47:14.000342 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:47:14.000342 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:47:14.000342 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:47:14.000342 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:47:14.000342 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 09:47:14.000342 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 09:47:14.000342 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 09:47:14.000342 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 09:47:14.000342 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:47:14.000342 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:47:14.000342 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:47:14.000342 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:47:14.000342 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:47:14.001199 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:47:14.001199 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:47:14.001199 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:47:14.001199 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:47:14.001199 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:47:14.001199 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:47:14.001199 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:47:14.001199 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:47:14.001199 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:47:14.001199 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:47:14.001199 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:47:14.001199 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 09:47:14.001199 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 09:47:14.001199 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 02 09:47:14.001199 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:47:14.001199 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 455/921] 10.222.0.64 () {56 vars in 1132 bytes} [Tue Aug 2 09:47:13 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:47:14.007097 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b638ab1d-1f25-4638-a55f-83a8be14e5e3 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:14.112179 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b638ab1d-1f25-4638-a55f-83a8be14e5e3 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:14.123474 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:14.151166 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d076d70-f9f2-48a5-ad5e-8a7ba9052b17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:14.151792 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d076d70-f9f2-48a5-ad5e-8a7ba9052b17 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:14.152077 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d076d70-f9f2-48a5-ad5e-8a7ba9052b17 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:14.152388 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d076d70-f9f2-48a5-ad5e-8a7ba9052b17 None admin] PATH_INFO: `/v3/services` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:14.153081 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d076d70-f9f2-48a5-ad5e-8a7ba9052b17 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:47:14.153389 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d076d70-f9f2-48a5-ad5e-8a7ba9052b17 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:14.154775 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d076d70-f9f2-48a5-ad5e-8a7ba9052b17 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:14.160970 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 467/922] 10.222.0.64 () {56 vars in 1126 bytes} [Tue Aug 2 09:47:14 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:47:14.169360 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af7752c-cec5-4d10-84f6-c4c0dbec4b2b None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:14.282367 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af7752c-cec5-4d10-84f6-c4c0dbec4b2b None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:14.304377 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:14.321682 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8360397-1c06-4567-9186-0b5d7b651206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:14.322302 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8360397-1c06-4567-9186-0b5d7b651206 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:14.322648 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8360397-1c06-4567-9186-0b5d7b651206 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:14.322927 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8360397-1c06-4567-9186-0b5d7b651206 None admin] PATH_INFO: `/v3/services` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:14.323621 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8360397-1c06-4567-9186-0b5d7b651206 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:47:14.323924 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8360397-1c06-4567-9186-0b5d7b651206 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:14.325326 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8360397-1c06-4567-9186-0b5d7b651206 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:14.335765 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 456/923] 10.222.0.64 () {56 vars in 1126 bytes} [Tue Aug 2 09:47:14 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:47:14.337209 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d076d70-f9f2-48a5-ad5e-8a7ba9052b17 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:14.448908 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d076d70-f9f2-48a5-ad5e-8a7ba9052b17 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:14.458444 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9769709-741e-4b42-b99a-3b230102e926 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:14.458826 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9769709-741e-4b42-b99a-3b230102e926 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:14.458998 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9769709-741e-4b42-b99a-3b230102e926 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:14.460619 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 457/924] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:47:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 09:47:14.465469 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f350d553-5f50-44ca-b2e8-85c48273a2f2 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:14.465766 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f350d553-5f50-44ca-b2e8-85c48273a2f2 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:14.466035 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f350d553-5f50-44ca-b2e8-85c48273a2f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:14.485086 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-f350d553-5f50-44ca-b2e8-85c48273a2f2 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:47:14.486278 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-f350d553-5f50-44ca-b2e8-85c48273a2f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:47:14.490109 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88c6077-dca7-4c3c-a13f-ea3819be1f33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:14.490726 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88c6077-dca7-4c3c-a13f-ea3819be1f33 None admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:14.491009 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88c6077-dca7-4c3c-a13f-ea3819be1f33 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:14.491283 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88c6077-dca7-4c3c-a13f-ea3819be1f33 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:14.492010 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88c6077-dca7-4c3c-a13f-ea3819be1f33 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:14.493408 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88c6077-dca7-4c3c-a13f-ea3819be1f33 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:14.519728 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 468/925] 10.222.0.64 () {60 vars in 1144 bytes} [Tue Aug 2 09:47:14 2022] POST /identity/v3/endpoints => generated 347 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:47:14.521670 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1f2508-2ce6-45b4-8f82-1f7f01d7afa5 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:14.521956 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1f2508-2ce6-45b4-8f82-1f7f01d7afa5 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:14.522283 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1f2508-2ce6-45b4-8f82-1f7f01d7afa5 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:14.523792 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 469/926] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:47:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:47:14.530913 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08850caf-454f-43f0-bb43-df4ecd3d4eaf None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:14.531302 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08850caf-454f-43f0-bb43-df4ecd3d4eaf None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:14.531469 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08850caf-454f-43f0-bb43-df4ecd3d4eaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:14.560222 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-08850caf-454f-43f0-bb43-df4ecd3d4eaf None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:47:14.562414 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-08850caf-454f-43f0-bb43-df4ecd3d4eaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:47:14.586489 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 458/927] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:47:14 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 09:47:14.593608 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:14.613483 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f350d553-5f50-44ca-b2e8-85c48273a2f2 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:14.652710 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 470/928] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:47:14 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 09:47:14.676401 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08850caf-454f-43f0-bb43-df4ecd3d4eaf None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:14.733618 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f350d553-5f50-44ca-b2e8-85c48273a2f2 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:14.746600 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:14.783627 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dd4368-2674-4465-9f91-c5bc52d1b689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:14.784164 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dd4368-2674-4465-9f91-c5bc52d1b689 None admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:14.784492 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dd4368-2674-4465-9f91-c5bc52d1b689 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:14.784771 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dd4368-2674-4465-9f91-c5bc52d1b689 None admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:14.785455 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dd4368-2674-4465-9f91-c5bc52d1b689 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:14.786971 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dd4368-2674-4465-9f91-c5bc52d1b689 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:14.789639 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: WARNING keystone.server.flask.common [None req-94dd4368-2674-4465-9f91-c5bc52d1b689 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 02 09:47:14.797624 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08850caf-454f-43f0-bb43-df4ecd3d4eaf None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:14.809525 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.sql.core [None req-94dd4368-2674-4465-9f91-c5bc52d1b689 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 02 09:47:14.809525 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 02 09:47:14.809525 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [parameters: {'id': 'dc51e51ad3104f4dac4a01344af9e362', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 02 09:47:14.809525 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80154) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 02 09:47:14.835825 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-94dd4368-2674-4465-9f91-c5bc52d1b689 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id dc51e51ad3104f4dac4a01344af9e362.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id dc51e51ad3104f4dac4a01344af9e362. Aug 02 09:47:14.835825 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:47:14.835825 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 02 09:47:14.835825 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 02 09:47:14.835825 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 02 09:47:14.835825 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 02 09:47:14.835825 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 02 09:47:14.835825 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application result = self._query(query) Aug 02 09:47:14.835825 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 02 09:47:14.835825 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application conn.query(q) Aug 02 09:47:14.835825 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 02 09:47:14.835825 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 02 09:47:14.835825 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 02 09:47:14.836738 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application result.read() Aug 02 09:47:14.836738 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 02 09:47:14.836738 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 02 09:47:14.836738 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 02 09:47:14.836738 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 02 09:47:14.836738 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 02 09:47:14.836738 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 02 09:47:14.836738 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 02 09:47:14.836738 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 02 09:47:14.836738 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 02 09:47:14.836738 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:47:14.836738 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 02 09:47:14.836738 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:47:14.836738 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:47:14.836738 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 02 09:47:14.836738 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 02 09:47:14.836738 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 02 09:47:14.836738 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 02 09:47:14.837730 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 02 09:47:14.837730 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application next(self.gen) Aug 02 09:47:14.837730 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 02 09:47:14.837730 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application yield resource Aug 02 09:47:14.837730 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 02 09:47:14.837730 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application next(self.gen) Aug 02 09:47:14.837730 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 02 09:47:14.837730 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application self.session.rollback() Aug 02 09:47:14.837730 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 02 09:47:14.837730 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application self.force_reraise() Aug 02 09:47:14.837730 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 02 09:47:14.837730 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise self.value Aug 02 09:47:14.837730 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 02 09:47:14.837730 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 02 09:47:14.837730 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 02 09:47:14.837730 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application session.commit() Aug 02 09:47:14.837730 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 02 09:47:14.838662 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 02 09:47:14.838662 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 02 09:47:14.838662 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application self._prepare_impl() Aug 02 09:47:14.838662 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 02 09:47:14.838662 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application self.session.flush() Aug 02 09:47:14.838662 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 02 09:47:14.838662 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application self._flush(objects) Aug 02 09:47:14.838662 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 02 09:47:14.838662 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 02 09:47:14.838662 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 02 09:47:14.838662 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application compat.raise_( Aug 02 09:47:14.838662 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 02 09:47:14.838662 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception Aug 02 09:47:14.838662 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 02 09:47:14.838662 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application flush_context.execute() Aug 02 09:47:14.838662 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 02 09:47:14.838662 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rec.execute(self) Aug 02 09:47:14.839619 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 02 09:47:14.839619 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 02 09:47:14.839619 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 02 09:47:14.839619 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 02 09:47:14.839619 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 02 09:47:14.839619 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 02 09:47:14.839619 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 02 09:47:14.839619 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 02 09:47:14.839619 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 02 09:47:14.839619 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 02 09:47:14.839619 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 02 09:47:14.839619 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 02 09:47:14.839619 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 02 09:47:14.839619 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 02 09:47:14.839619 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 02 09:47:14.845703 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 02 09:47:14.845703 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 02 09:47:14.845703 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception Aug 02 09:47:14.845703 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 02 09:47:14.845703 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 02 09:47:14.845703 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 02 09:47:14.845703 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 02 09:47:14.845703 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 02 09:47:14.845703 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application result = self._query(query) Aug 02 09:47:14.845703 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 02 09:47:14.845703 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application conn.query(q) Aug 02 09:47:14.845703 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 02 09:47:14.845703 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 02 09:47:14.845703 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 02 09:47:14.845703 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application result.read() Aug 02 09:47:14.845703 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 02 09:47:14.845703 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 02 09:47:14.851532 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 02 09:47:14.851532 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 02 09:47:14.851532 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 02 09:47:14.851532 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 02 09:47:14.851532 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 02 09:47:14.851532 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 02 09:47:14.851532 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 02 09:47:14.851532 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 02 09:47:14.851532 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [parameters: {'id': 'dc51e51ad3104f4dac4a01344af9e362', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 02 09:47:14.851532 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 02 09:47:14.851532 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:47:14.851532 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 09:47:14.851532 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:47:14.851532 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:47:14.851532 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 02 09:47:14.860871 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 02 09:47:14.860871 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 02 09:47:14.860871 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 02 09:47:14.860871 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 02 09:47:14.860871 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:47:14.860871 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 09:47:14.860871 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:47:14.860871 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:47:14.860871 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:47:14.860871 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:47:14.860871 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:47:14.860871 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:47:14.860871 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:47:14.860871 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:47:14.860871 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:47:14.860871 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:47:14.860871 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:47:14.864039 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:47:14.864039 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 02 09:47:14.864039 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 02 09:47:14.864039 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:47:14.864039 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:47:14.864039 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 02 09:47:14.864039 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 02 09:47:14.864039 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id dc51e51ad3104f4dac4a01344af9e362. Aug 02 09:47:14.864039 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:47:14.864039 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 459/929] 10.222.0.64 () {60 vars in 1140 bytes} [Tue Aug 2 09:47:14 2022] POST /identity/v3/projects => generated 262 bytes in 218 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 02 09:47:14.864039 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dd4368-2674-4465-9f91-c5bc52d1b689 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:14.866166 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ecdc34-514a-4c1c-ab13-cfad1d5d8b86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:14.866166 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ecdc34-514a-4c1c-ab13-cfad1d5d8b86 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:14.866166 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ecdc34-514a-4c1c-ab13-cfad1d5d8b86 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:14.866166 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ecdc34-514a-4c1c-ab13-cfad1d5d8b86 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:14.867011 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ecdc34-514a-4c1c-ab13-cfad1d5d8b86 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:14.867011 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ecdc34-514a-4c1c-ab13-cfad1d5d8b86 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:14.873523 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-98ecdc34-514a-4c1c-ab13-cfad1d5d8b86 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 02 09:47:14.873523 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:47:14.873523 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 09:47:14.873523 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 09:47:14.873523 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 09:47:14.873523 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 09:47:14.873523 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 09:47:14.873523 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 09:47:14.873523 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 02 09:47:14.873523 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:47:14.873523 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 09:47:14.873523 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:47:14.873523 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:47:14.873523 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:47:14.873523 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:47:14.873523 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:47:14.873523 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:47:14.876268 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:47:14.876268 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:47:14.876268 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:47:14.876268 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:47:14.876268 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:47:14.876268 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:47:14.876268 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 09:47:14.876268 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 09:47:14.876268 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 09:47:14.876268 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 09:47:14.876268 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:47:14.876268 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:47:14.876268 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:47:14.876268 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:47:14.876268 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:47:14.876268 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:47:14.878358 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:47:14.878358 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:47:14.878358 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:47:14.878358 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:47:14.878358 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:47:14.878358 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:47:14.878358 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:47:14.878358 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:47:14.878358 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:47:14.878358 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:47:14.878358 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 09:47:14.878358 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 09:47:14.878358 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 02 09:47:14.878358 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:47:14.878358 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 471/930] 10.222.0.64 () {56 vars in 1126 bytes} [Tue Aug 2 09:47:14 2022] GET /identity/v3/services/object-store => generated 93 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 09:47:14.889631 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ecdc34-514a-4c1c-ab13-cfad1d5d8b86 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:14.922302 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:14.982850 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dd4368-2674-4465-9f91-c5bc52d1b689 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:15.008870 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ecdc34-514a-4c1c-ab13-cfad1d5d8b86 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:15.024507 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d3c279-656a-46f5-ab5b-93c44b565d15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:15.025223 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d3c279-656a-46f5-ab5b-93c44b565d15 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:15.025507 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d3c279-656a-46f5-ab5b-93c44b565d15 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:15.025781 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d3c279-656a-46f5-ab5b-93c44b565d15 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:15.033776 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d3c279-656a-46f5-ab5b-93c44b565d15 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:15.035443 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d3c279-656a-46f5-ab5b-93c44b565d15 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:15.050230 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-d4d3c279-656a-46f5-ab5b-93c44b565d15 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 09:47:15.050230 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:47:15.050230 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:47:15.050230 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:47:15.050230 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:47:15.050230 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:47:15.050230 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:47:15.050230 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:47:15.050230 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:47:15.050230 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:47:15.050230 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:47:15.050230 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:47:15.050230 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 09:47:15.050230 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 09:47:15.050230 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 09:47:15.051215 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 09:47:15.051215 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:47:15.051215 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:47:15.051215 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:47:15.051215 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:47:15.051215 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:47:15.051215 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:47:15.051215 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:47:15.051215 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:47:15.051215 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:47:15.051215 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:47:15.051215 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:47:15.051215 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:47:15.051215 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:47:15.051215 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:47:15.051215 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:47:15.051215 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:47:15.052128 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 09:47:15.052128 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 09:47:15.052128 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 09:47:15.052128 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 09:47:15.052128 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 09:47:15.052128 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 09:47:15.052128 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 09:47:15.052128 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:47:15.053277 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d302e85-a728-4d95-80bc-21e16240e964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:15.054166 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d302e85-a728-4d95-80bc-21e16240e964 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:15.054666 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d302e85-a728-4d95-80bc-21e16240e964 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:15.055148 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d302e85-a728-4d95-80bc-21e16240e964 None admin] PATH_INFO: `/v3/services` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:15.056061 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d302e85-a728-4d95-80bc-21e16240e964 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:47:15.056594 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d302e85-a728-4d95-80bc-21e16240e964 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:15.058133 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 460/931] 10.222.0.64 () {56 vars in 1111 bytes} [Tue Aug 2 09:47:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 09:47:15.058416 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d302e85-a728-4d95-80bc-21e16240e964 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:15.059554 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d3c279-656a-46f5-ab5b-93c44b565d15 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:15.064935 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 472/932] 10.222.0.64 () {56 vars in 1122 bytes} [Tue Aug 2 09:47:14 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:47:15.071307 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d302e85-a728-4d95-80bc-21e16240e964 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:15.094133 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:15.176977 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:15.178299 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d3c279-656a-46f5-ab5b-93c44b565d15 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:15.198741 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d302e85-a728-4d95-80bc-21e16240e964 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:15.216199 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af60e899-a50f-4261-9581-6a0da2527a1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:15.216852 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af60e899-a50f-4261-9581-6a0da2527a1a None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:15.217146 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af60e899-a50f-4261-9581-6a0da2527a1a None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:15.217401 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af60e899-a50f-4261-9581-6a0da2527a1a None admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:15.218865 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af60e899-a50f-4261-9581-6a0da2527a1a None admin] RBAC: Adding query filter params (name=service) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:47:15.219174 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af60e899-a50f-4261-9581-6a0da2527a1a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:15.220957 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af60e899-a50f-4261-9581-6a0da2527a1a None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:15.239845 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba49e61-c99d-4cad-97d3-ea24e42fa8d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:15.241768 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba49e61-c99d-4cad-97d3-ea24e42fa8d5 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:15.242256 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba49e61-c99d-4cad-97d3-ea24e42fa8d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:15.246222 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba49e61-c99d-4cad-97d3-ea24e42fa8d5 None admin] PATH_INFO: `/v3/services` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:15.247254 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba49e61-c99d-4cad-97d3-ea24e42fa8d5 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:47:15.247797 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba49e61-c99d-4cad-97d3-ea24e42fa8d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:15.249548 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba49e61-c99d-4cad-97d3-ea24e42fa8d5 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:15.260924 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 461/934] 10.222.0.64 () {56 vars in 1112 bytes} [Tue Aug 2 09:47:15 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:47:15.261677 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 473/934] 10.222.0.64 () {56 vars in 1122 bytes} [Tue Aug 2 09:47:15 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:47:15.262646 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af60e899-a50f-4261-9581-6a0da2527a1a None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:15.373710 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:15.394109 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af60e899-a50f-4261-9581-6a0da2527a1a None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:15.462131 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36481506-b5cb-4bc5-ac60-0a3715ff8b4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:15.463208 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36481506-b5cb-4bc5-ac60-0a3715ff8b4b None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:15.463711 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36481506-b5cb-4bc5-ac60-0a3715ff8b4b None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:15.464204 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36481506-b5cb-4bc5-ac60-0a3715ff8b4b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:15.465707 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36481506-b5cb-4bc5-ac60-0a3715ff8b4b None admin] RBAC: Adding query filter params (interface=public, service_id=7a1ff592a4444f2880101f5890b14962, region_id=RegionOne) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:47:15.466284 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36481506-b5cb-4bc5-ac60-0a3715ff8b4b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:15.469701 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36481506-b5cb-4bc5-ac60-0a3715ff8b4b None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:15.481681 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 462/935] 10.222.0.64 () {56 vars in 1251 bytes} [Tue Aug 2 09:47:15 2022] GET /identity/v3/endpoints?service_id=7a1ff592a4444f2880101f5890b14962&interface=public®ion_id=RegionOne => generated 194 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:47:15.487077 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba49e61-c99d-4cad-97d3-ea24e42fa8d5 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:15.605112 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba49e61-c99d-4cad-97d3-ea24e42fa8d5 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:15.606637 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f8c86d-db70-4f81-be54-d487117754c7 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:15.606932 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f8c86d-db70-4f81-be54-d487117754c7 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:15.607217 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f8c86d-db70-4f81-be54-d487117754c7 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:15.609442 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 463/936] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:47:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:47:15.614641 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809d4b61-2950-4728-b4c8-944c3b15451c None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:15.614925 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809d4b61-2950-4728-b4c8-944c3b15451c None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:15.615201 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809d4b61-2950-4728-b4c8-944c3b15451c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:15.640461 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-809d4b61-2950-4728-b4c8-944c3b15451c None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:47:15.642927 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-809d4b61-2950-4728-b4c8-944c3b15451c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:47:15.644986 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fd0d5a-04d9-4bc6-aed7-f65fdfe5cdb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:15.645632 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fd0d5a-04d9-4bc6-aed7-f65fdfe5cdb9 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:15.645925 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fd0d5a-04d9-4bc6-aed7-f65fdfe5cdb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:15.646202 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fd0d5a-04d9-4bc6-aed7-f65fdfe5cdb9 None admin] PATH_INFO: `/v3/services` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:15.646872 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fd0d5a-04d9-4bc6-aed7-f65fdfe5cdb9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:15.648274 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fd0d5a-04d9-4bc6-aed7-f65fdfe5cdb9 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:15.660038 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 474/937] 10.222.0.64 () {56 vars in 1087 bytes} [Tue Aug 2 09:47:15 2022] GET /identity/v3/services => generated 1497 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:47:15.741325 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 464/938] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:47:15 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 09:47:15.763254 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fd0d5a-04d9-4bc6-aed7-f65fdfe5cdb9 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:15.785326 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:15.880073 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fd0d5a-04d9-4bc6-aed7-f65fdfe5cdb9 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:15.915556 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974c1112-f66c-474d-a683-21df468fb970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:15.916125 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974c1112-f66c-474d-a683-21df468fb970 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:15.916484 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974c1112-f66c-474d-a683-21df468fb970 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:15.916728 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974c1112-f66c-474d-a683-21df468fb970 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:15.927923 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974c1112-f66c-474d-a683-21df468fb970 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 09:47:15.933290 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-974c1112-f66c-474d-a683-21df468fb970 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 02 09:47:15.933290 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:47:15.933290 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:47:15.933290 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:47:15.933290 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:47:15.933290 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:47:15.933290 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:47:15.933290 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:47:15.933290 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:47:15.933290 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:47:15.933290 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:47:15.933290 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:47:15.933290 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 09:47:15.933290 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 09:47:15.933290 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 09:47:15.933290 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 09:47:15.934225 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 09:47:15.934225 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 09:47:15.934225 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 09:47:15.934225 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 09:47:15.934225 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:47:15.934225 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:47:15.934225 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:47:15.934225 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:47:15.934225 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:47:15.934225 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:47:15.934225 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:47:15.934225 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:47:15.934225 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:47:15.934225 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:47:15.934225 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:47:15.934225 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:47:15.935097 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:47:15.935097 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:47:15.935097 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:47:15.935097 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:47:15.935097 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 09:47:15.935097 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 09:47:15.935097 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 09:47:15.935097 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 09:47:15.935097 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 09:47:15.935097 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 09:47:15.935097 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 02 09:47:15.935097 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:47:15.935097 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 475/939] 10.222.0.64 () {56 vars in 1120 bytes} [Tue Aug 2 09:47:15 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:47:15.941203 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809d4b61-2950-4728-b4c8-944c3b15451c None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:16.024450 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:16.044354 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809d4b61-2950-4728-b4c8-944c3b15451c None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:16.079338 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecea01aa-dbc1-41ef-a862-5b61a951d6d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:16.080004 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecea01aa-dbc1-41ef-a862-5b61a951d6d5 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:16.080312 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecea01aa-dbc1-41ef-a862-5b61a951d6d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:16.080648 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecea01aa-dbc1-41ef-a862-5b61a951d6d5 None admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:16.081358 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecea01aa-dbc1-41ef-a862-5b61a951d6d5 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:47:16.081662 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecea01aa-dbc1-41ef-a862-5b61a951d6d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:16.083100 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecea01aa-dbc1-41ef-a862-5b61a951d6d5 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:16.096259 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 465/940] 10.222.0.64 () {56 vars in 1115 bytes} [Tue Aug 2 09:47:15 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:47:16.101707 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974c1112-f66c-474d-a683-21df468fb970 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:16.181350 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:16.208422 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974c1112-f66c-474d-a683-21df468fb970 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:16.244081 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729296d3-66f0-478b-a184-1d06979a36b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:16.244763 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729296d3-66f0-478b-a184-1d06979a36b5 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:16.245051 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729296d3-66f0-478b-a184-1d06979a36b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:16.245333 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729296d3-66f0-478b-a184-1d06979a36b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:16.256829 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729296d3-66f0-478b-a184-1d06979a36b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:16.258315 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729296d3-66f0-478b-a184-1d06979a36b5 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:16.272989 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-729296d3-66f0-478b-a184-1d06979a36b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:47:16.272989 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:47:16.272989 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:47:16.272989 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:47:16.272989 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:47:16.272989 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:47:16.272989 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:47:16.272989 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:47:16.272989 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:47:16.272989 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:47:16.272989 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:47:16.272989 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:47:16.272989 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 09:47:16.272989 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 09:47:16.272989 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 09:47:16.273932 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 09:47:16.273932 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:47:16.273932 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:47:16.273932 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:47:16.273932 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:47:16.273932 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:47:16.273932 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:47:16.273932 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:47:16.273932 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:47:16.273932 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:47:16.273932 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:47:16.273932 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:47:16.273932 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:47:16.273932 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:47:16.273932 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:47:16.273932 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:47:16.273932 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:47:16.274840 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 09:47:16.274840 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 09:47:16.274840 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:47:16.274840 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:47:16.274840 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 476/941] 10.222.0.64 () {56 vars in 1108 bytes} [Tue Aug 2 09:47:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 09:47:16.281802 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecea01aa-dbc1-41ef-a862-5b61a951d6d5 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:16.385713 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecea01aa-dbc1-41ef-a862-5b61a951d6d5 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:16.433745 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887e1670-bde3-4e71-8748-6c225f451f19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:16.434296 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887e1670-bde3-4e71-8748-6c225f451f19 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:16.434587 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887e1670-bde3-4e71-8748-6c225f451f19 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:16.434865 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887e1670-bde3-4e71-8748-6c225f451f19 None admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:16.435577 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887e1670-bde3-4e71-8748-6c225f451f19 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:47:16.435888 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887e1670-bde3-4e71-8748-6c225f451f19 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:16.437341 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887e1670-bde3-4e71-8748-6c225f451f19 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:16.451525 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 466/942] 10.222.0.64 () {56 vars in 1109 bytes} [Tue Aug 2 09:47:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:47:16.460758 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729296d3-66f0-478b-a184-1d06979a36b5 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:16.641436 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729296d3-66f0-478b-a184-1d06979a36b5 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:16.688014 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:16.700668 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362424c9-5099-45da-857e-18be1d36648c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:16.701321 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362424c9-5099-45da-857e-18be1d36648c None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:16.701603 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362424c9-5099-45da-857e-18be1d36648c None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:16.701878 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362424c9-5099-45da-857e-18be1d36648c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:16.710428 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362424c9-5099-45da-857e-18be1d36648c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:16.711930 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362424c9-5099-45da-857e-18be1d36648c None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:16.724405 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-362424c9-5099-45da-857e-18be1d36648c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 02 09:47:16.724405 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:47:16.724405 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:47:16.724405 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:47:16.724405 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:47:16.724405 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:47:16.724405 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:47:16.724405 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:47:16.724405 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:47:16.724405 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:47:16.724405 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:47:16.724405 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:47:16.724405 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 09:47:16.724405 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 09:47:16.724405 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 09:47:16.724405 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 09:47:16.725380 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:47:16.725380 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:47:16.725380 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 09:47:16.725380 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 09:47:16.725380 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 09:47:16.725380 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 09:47:16.725380 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:47:16.725380 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:47:16.725380 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:47:16.725380 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:47:16.725380 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:47:16.725380 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:47:16.725380 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:47:16.725380 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:47:16.725380 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:47:16.725380 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:47:16.725380 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:47:16.725380 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:47:16.726351 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:47:16.726351 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:47:16.726351 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 09:47:16.726351 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 09:47:16.726351 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 09:47:16.726351 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 09:47:16.726351 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 09:47:16.726351 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 09:47:16.726351 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 02 09:47:16.726351 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:47:16.726351 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 477/943] 10.222.0.64 () {56 vars in 1117 bytes} [Tue Aug 2 09:47:16 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:47:16.731805 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887e1670-bde3-4e71-8748-6c225f451f19 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:16.735265 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:16.752049 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b60c671-4b8c-4301-aebb-200740f60603 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:16.752409 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b60c671-4b8c-4301-aebb-200740f60603 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:16.752645 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b60c671-4b8c-4301-aebb-200740f60603 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:16.754112 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 478/944] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:47:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 09:47:16.759341 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc35131-d3ee-4df5-8b7a-05dd88581a8f None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:16.759825 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc35131-d3ee-4df5-8b7a-05dd88581a8f None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:16.760339 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc35131-d3ee-4df5-8b7a-05dd88581a8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:16.788304 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-bfc35131-d3ee-4df5-8b7a-05dd88581a8f None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:47:16.788304 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-bfc35131-d3ee-4df5-8b7a-05dd88581a8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:47:16.878553 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887e1670-bde3-4e71-8748-6c225f451f19 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:16.881458 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 479/945] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:47:16 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 02 09:47:16.903087 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc35131-d3ee-4df5-8b7a-05dd88581a8f None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:16.925822 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcc404e-3c8b-459e-870a-1c06993c26b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:16.926712 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcc404e-3c8b-459e-870a-1c06993c26b2 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:16.927030 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcc404e-3c8b-459e-870a-1c06993c26b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:16.927671 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcc404e-3c8b-459e-870a-1c06993c26b2 None admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:16.928528 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcc404e-3c8b-459e-870a-1c06993c26b2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:47:16.928844 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcc404e-3c8b-459e-870a-1c06993c26b2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:16.930325 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcc404e-3c8b-459e-870a-1c06993c26b2 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:16.955793 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 467/946] 10.222.0.64 () {56 vars in 1149 bytes} [Tue Aug 2 09:47:16 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:47:16.962423 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcc404e-3c8b-459e-870a-1c06993c26b2 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:17.036559 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc35131-d3ee-4df5-8b7a-05dd88581a8f None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:17.049826 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:17.079421 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:17.103667 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c6d9fe-d6a1-42e0-adc9-84ef85d38f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:17.104452 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c6d9fe-d6a1-42e0-adc9-84ef85d38f36 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:17.104776 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c6d9fe-d6a1-42e0-adc9-84ef85d38f36 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:17.105060 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c6d9fe-d6a1-42e0-adc9-84ef85d38f36 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:17.105920 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c6d9fe-d6a1-42e0-adc9-84ef85d38f36 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:17.107266 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c6d9fe-d6a1-42e0-adc9-84ef85d38f36 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:17.112324 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcc404e-3c8b-459e-870a-1c06993c26b2 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:17.117315 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-19c6d9fe-d6a1-42e0-adc9-84ef85d38f36 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 02 09:47:17.117315 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:47:17.117315 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 09:47:17.117315 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 09:47:17.117315 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 09:47:17.117315 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 09:47:17.117315 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 09:47:17.117315 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 09:47:17.117315 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 02 09:47:17.117315 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:47:17.117315 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 09:47:17.117315 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:47:17.117315 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:47:17.117315 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:47:17.117315 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:47:17.117315 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:47:17.117315 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:47:17.118593 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:47:17.118593 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:47:17.118593 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:47:17.118593 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:47:17.118593 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:47:17.118593 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:47:17.118593 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 09:47:17.118593 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 09:47:17.118593 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 09:47:17.118593 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 09:47:17.118593 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:47:17.118593 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:47:17.118593 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:47:17.118593 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:47:17.118593 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:47:17.118593 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:47:17.119703 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:47:17.119703 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:47:17.119703 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:47:17.119703 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:47:17.119703 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:47:17.119703 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:47:17.119703 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:47:17.119703 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:47:17.119703 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:47:17.119703 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:47:17.119703 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 09:47:17.119703 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 09:47:17.119703 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 02 09:47:17.119703 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:47:17.123463 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 480/947] 10.222.0.64 () {56 vars in 1111 bytes} [Tue Aug 2 09:47:16 2022] GET /identity/v3/services/compute => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:47:17.125186 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c6d9fe-d6a1-42e0-adc9-84ef85d38f36 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:17.154778 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c92c4e-a275-4f4c-a167-953bf1157713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:17.155834 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c92c4e-a275-4f4c-a167-953bf1157713 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:17.156481 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c92c4e-a275-4f4c-a167-953bf1157713 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:17.156940 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c92c4e-a275-4f4c-a167-953bf1157713 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:17.168152 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c92c4e-a275-4f4c-a167-953bf1157713 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:17.170028 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c92c4e-a275-4f4c-a167-953bf1157713 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:17.187057 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-96c92c4e-a275-4f4c-a167-953bf1157713 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:47:17.187057 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:47:17.187057 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:47:17.187057 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:47:17.187057 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:47:17.187057 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:47:17.187057 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:47:17.187057 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:47:17.187057 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:47:17.187057 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:47:17.187057 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:47:17.187057 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:47:17.187057 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 09:47:17.187057 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 09:47:17.187057 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 09:47:17.188061 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 09:47:17.188061 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:47:17.188061 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:47:17.188061 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:47:17.188061 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:47:17.188061 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:47:17.188061 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:47:17.188061 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:47:17.188061 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:47:17.188061 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:47:17.188061 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:47:17.188061 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:47:17.188061 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:47:17.188061 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:47:17.188061 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:47:17.188061 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:47:17.188061 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:47:17.189027 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 09:47:17.189027 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 09:47:17.189027 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:47:17.189027 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:47:17.191432 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 468/948] 10.222.0.64 () {56 vars in 1108 bytes} [Tue Aug 2 09:47:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:47:17.199417 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c92c4e-a275-4f4c-a167-953bf1157713 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:17.235252 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c6d9fe-d6a1-42e0-adc9-84ef85d38f36 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:17.274141 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bdb059-b7ed-4f3f-8650-d7289487b258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:17.275180 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bdb059-b7ed-4f3f-8650-d7289487b258 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:17.275682 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bdb059-b7ed-4f3f-8650-d7289487b258 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:17.276345 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bdb059-b7ed-4f3f-8650-d7289487b258 None admin] PATH_INFO: `/v3/services` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:17.277809 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bdb059-b7ed-4f3f-8650-d7289487b258 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:47:17.278448 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bdb059-b7ed-4f3f-8650-d7289487b258 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:17.281479 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bdb059-b7ed-4f3f-8650-d7289487b258 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:17.293697 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 481/949] 10.222.0.64 () {56 vars in 1112 bytes} [Tue Aug 2 09:47:17 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:47:17.294751 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bdb059-b7ed-4f3f-8650-d7289487b258 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:17.315803 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c92c4e-a275-4f4c-a167-953bf1157713 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:17.333238 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:17.344618 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:17.361661 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01913b42-bb42-42af-bf76-a9abdf29a905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:17.362326 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01913b42-bb42-42af-bf76-a9abdf29a905 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:17.362616 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01913b42-bb42-42af-bf76-a9abdf29a905 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:17.362938 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01913b42-bb42-42af-bf76-a9abdf29a905 None admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:17.363830 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01913b42-bb42-42af-bf76-a9abdf29a905 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:47:17.364154 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01913b42-bb42-42af-bf76-a9abdf29a905 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:17.365562 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01913b42-bb42-42af-bf76-a9abdf29a905 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:17.383770 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 469/950] 10.222.0.64 () {56 vars in 1109 bytes} [Tue Aug 2 09:47:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:47:17.385036 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01913b42-bb42-42af-bf76-a9abdf29a905 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:17.404681 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bdb059-b7ed-4f3f-8650-d7289487b258 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:17.441993 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e97257-becd-4d04-9c24-ce52a9c10e53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:17.442700 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e97257-becd-4d04-9c24-ce52a9c10e53 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:17.443084 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e97257-becd-4d04-9c24-ce52a9c10e53 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:17.443366 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e97257-becd-4d04-9c24-ce52a9c10e53 None admin] PATH_INFO: `/v3/services` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:17.444050 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e97257-becd-4d04-9c24-ce52a9c10e53 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:47:17.444452 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e97257-becd-4d04-9c24-ce52a9c10e53 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:17.445824 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e97257-becd-4d04-9c24-ce52a9c10e53 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:17.456676 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 482/951] 10.222.0.64 () {56 vars in 1112 bytes} [Tue Aug 2 09:47:17 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:47:17.502294 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01913b42-bb42-42af-bf76-a9abdf29a905 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:17.541665 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4f231a-44f8-4e18-a016-a539a4b1e3c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:17.542339 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4f231a-44f8-4e18-a016-a539a4b1e3c0 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:17.542629 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4f231a-44f8-4e18-a016-a539a4b1e3c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:17.542912 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4f231a-44f8-4e18-a016-a539a4b1e3c0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:17.551658 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4f231a-44f8-4e18-a016-a539a4b1e3c0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:17.553192 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4f231a-44f8-4e18-a016-a539a4b1e3c0 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:17.567213 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-ab4f231a-44f8-4e18-a016-a539a4b1e3c0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 09:47:17.567213 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:47:17.567213 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:47:17.567213 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:47:17.567213 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:47:17.567213 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:47:17.567213 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:47:17.567213 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:47:17.567213 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:47:17.567213 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:47:17.567213 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:47:17.567213 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:47:17.567213 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 09:47:17.567213 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 09:47:17.567213 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 09:47:17.569878 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 09:47:17.569878 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:47:17.569878 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:47:17.569878 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:47:17.569878 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:47:17.569878 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:47:17.569878 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:47:17.569878 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:47:17.569878 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:47:17.569878 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:47:17.569878 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:47:17.569878 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:47:17.569878 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:47:17.569878 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:47:17.569878 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:47:17.569878 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:47:17.569878 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:47:17.572316 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 09:47:17.572316 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 09:47:17.572316 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 09:47:17.572316 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 09:47:17.572316 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 09:47:17.572316 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 09:47:17.572316 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 09:47:17.572316 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:47:17.572316 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 470/952] 10.222.0.64 () {56 vars in 1111 bytes} [Tue Aug 2 09:47:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:47:17.576511 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e97257-becd-4d04-9c24-ce52a9c10e53 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:17.613072 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774f5bb1-c1e3-45c6-aa2f-232bdf7947c5 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:17.613592 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774f5bb1-c1e3-45c6-aa2f-232bdf7947c5 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:17.615144 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774f5bb1-c1e3-45c6-aa2f-232bdf7947c5 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:17.615411 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 471/953] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:47:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 09:47:17.623161 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333e7fb4-3c1f-4fbc-866a-d11854ccde5d None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:17.623500 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333e7fb4-3c1f-4fbc-866a-d11854ccde5d None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:17.623778 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333e7fb4-3c1f-4fbc-866a-d11854ccde5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:17.653731 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-333e7fb4-3c1f-4fbc-866a-d11854ccde5d None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:47:17.655617 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-333e7fb4-3c1f-4fbc-866a-d11854ccde5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:47:17.690478 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:17.708425 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e97257-becd-4d04-9c24-ce52a9c10e53 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:17.718951 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:17.744352 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 472/954] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:47:17 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 02 09:47:17.754680 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae72c31-42a5-4553-aa2b-15c96a8011a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:17.755236 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae72c31-42a5-4553-aa2b-15c96a8011a3 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:17.755582 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae72c31-42a5-4553-aa2b-15c96a8011a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:17.755868 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae72c31-42a5-4553-aa2b-15c96a8011a3 None admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:17.756663 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae72c31-42a5-4553-aa2b-15c96a8011a3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:47:17.756978 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae72c31-42a5-4553-aa2b-15c96a8011a3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:17.758425 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae72c31-42a5-4553-aa2b-15c96a8011a3 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:17.769306 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333e7fb4-3c1f-4fbc-866a-d11854ccde5d None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:17.775413 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 483/955] 10.222.0.64 () {56 vars in 1148 bytes} [Tue Aug 2 09:47:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:47:17.781180 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae72c31-42a5-4553-aa2b-15c96a8011a3 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:17.890148 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae72c31-42a5-4553-aa2b-15c96a8011a3 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:17.913344 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333e7fb4-3c1f-4fbc-866a-d11854ccde5d None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:17.938794 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcc7c32-34b5-4c13-afa5-c6712a5ad4de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:17.939469 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcc7c32-34b5-4c13-afa5-c6712a5ad4de None admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:17.939756 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcc7c32-34b5-4c13-afa5-c6712a5ad4de None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:17.940041 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcc7c32-34b5-4c13-afa5-c6712a5ad4de None admin] PATH_INFO: `/v3/projects/390bd7c4f91c4465b0c7ab0f9e436c4d/users/5338bc8c594e4bc9bc41019226bf8d87/roles/55b4c595b54f40c6b03ed7161e5258bf` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:17.945248 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcc7c32-34b5-4c13-afa5-c6712a5ad4de None admin] RBAC: Authorizing `identity:create_grant(project_id=390bd7c4f91c4465b0c7ab0f9e436c4d, user_id=5338bc8c594e4bc9bc41019226bf8d87, role_id=55b4c595b54f40c6b03ed7161e5258bf)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:17.947704 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcc7c32-34b5-4c13-afa5-c6712a5ad4de None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:17.965135 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7fe5c7-359b-4d09-88b8-68d1c45837ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:17.965815 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7fe5c7-359b-4d09-88b8-68d1c45837ed None admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:17.966210 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7fe5c7-359b-4d09-88b8-68d1c45837ed None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:17.966390 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7fe5c7-359b-4d09-88b8-68d1c45837ed None admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:17.967182 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7fe5c7-359b-4d09-88b8-68d1c45837ed None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:17.969347 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7fe5c7-359b-4d09-88b8-68d1c45837ed None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:17.969609 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 484/956] 10.222.0.64 () {58 vars in 1439 bytes} [Tue Aug 2 09:47:17 2022] PUT /identity/v3/projects/390bd7c4f91c4465b0c7ab0f9e436c4d/users/5338bc8c594e4bc9bc41019226bf8d87/roles/55b4c595b54f40c6b03ed7161e5258bf => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:47:17.978469 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48365198-7796-4077-a590-2c3fa8e3c5a3 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:17.978904 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48365198-7796-4077-a590-2c3fa8e3c5a3 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:17.979403 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48365198-7796-4077-a590-2c3fa8e3c5a3 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:17.981349 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 485/957] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:47:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 09:47:17.987477 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a38a10-05f1-44d4-b817-d84db6c68c47 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:17.987801 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a38a10-05f1-44d4-b817-d84db6c68c47 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:17.988041 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a38a10-05f1-44d4-b817-d84db6c68c47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:18.007662 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.sql.core [None req-8b7fe5c7-359b-4d09-88b8-68d1c45837ed None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 02 09:47:18.007662 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 02 09:47:18.007662 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [parameters: {'user_id': '130c707df7dd4bd39c9f240ec2191961', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 02 09:47:18.007662 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80154) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 02 09:47:18.009257 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-a6a38a10-05f1-44d4-b817-d84db6c68c47 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:47:18.010462 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-a6a38a10-05f1-44d4-b817-d84db6c68c47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:47:18.019318 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-8b7fe5c7-359b-4d09-88b8-68d1c45837ed None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 02 09:47:18.019318 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:47:18.019318 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 02 09:47:18.019318 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 02 09:47:18.019318 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 02 09:47:18.019318 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 02 09:47:18.019318 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 02 09:47:18.019318 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application result = self._query(query) Aug 02 09:47:18.019318 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 02 09:47:18.019318 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application conn.query(q) Aug 02 09:47:18.019318 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 02 09:47:18.019318 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 02 09:47:18.019318 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 02 09:47:18.019318 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application result.read() Aug 02 09:47:18.019318 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 02 09:47:18.020465 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 02 09:47:18.020465 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 02 09:47:18.020465 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 02 09:47:18.020465 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 02 09:47:18.020465 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 02 09:47:18.020465 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 02 09:47:18.020465 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 02 09:47:18.020465 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 02 09:47:18.020465 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:47:18.020465 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 02 09:47:18.020465 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:47:18.020465 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:47:18.020465 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 02 09:47:18.020465 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 02 09:47:18.020465 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 02 09:47:18.020465 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 02 09:47:18.020465 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 02 09:47:18.020465 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application next(self.gen) Aug 02 09:47:18.021932 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 02 09:47:18.021932 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application yield resource Aug 02 09:47:18.021932 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 02 09:47:18.021932 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application next(self.gen) Aug 02 09:47:18.021932 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 02 09:47:18.021932 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application self.session.rollback() Aug 02 09:47:18.021932 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 02 09:47:18.021932 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application self.force_reraise() Aug 02 09:47:18.021932 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 02 09:47:18.021932 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise self.value Aug 02 09:47:18.021932 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 02 09:47:18.021932 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 02 09:47:18.021932 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 02 09:47:18.021932 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application session.commit() Aug 02 09:47:18.021932 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 02 09:47:18.021932 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 02 09:47:18.021932 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 02 09:47:18.022941 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application self._prepare_impl() Aug 02 09:47:18.022941 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 02 09:47:18.022941 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application self.session.flush() Aug 02 09:47:18.022941 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 02 09:47:18.022941 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application self._flush(objects) Aug 02 09:47:18.022941 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 02 09:47:18.022941 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 02 09:47:18.022941 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 02 09:47:18.022941 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application compat.raise_( Aug 02 09:47:18.022941 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 02 09:47:18.022941 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception Aug 02 09:47:18.022941 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 02 09:47:18.022941 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application flush_context.execute() Aug 02 09:47:18.022941 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 02 09:47:18.022941 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rec.execute(self) Aug 02 09:47:18.022941 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 02 09:47:18.022941 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 02 09:47:18.023869 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 02 09:47:18.023869 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 02 09:47:18.023869 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 02 09:47:18.023869 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 02 09:47:18.023869 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 02 09:47:18.023869 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 02 09:47:18.023869 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 02 09:47:18.023869 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 02 09:47:18.023869 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 02 09:47:18.023869 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 02 09:47:18.023869 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 02 09:47:18.023869 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 02 09:47:18.023869 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 02 09:47:18.023869 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 02 09:47:18.023869 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 02 09:47:18.023869 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception Aug 02 09:47:18.026214 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 02 09:47:18.026214 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 02 09:47:18.026214 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 02 09:47:18.026214 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 02 09:47:18.026214 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 02 09:47:18.026214 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application result = self._query(query) Aug 02 09:47:18.026214 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 02 09:47:18.026214 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application conn.query(q) Aug 02 09:47:18.026214 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 02 09:47:18.026214 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 02 09:47:18.026214 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 02 09:47:18.026214 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application result.read() Aug 02 09:47:18.026214 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 02 09:47:18.026214 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 02 09:47:18.026214 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 02 09:47:18.026214 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 02 09:47:18.031526 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 02 09:47:18.031526 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 02 09:47:18.031526 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 02 09:47:18.031526 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 02 09:47:18.031526 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 02 09:47:18.031526 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 02 09:47:18.031526 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [parameters: {'user_id': '130c707df7dd4bd39c9f240ec2191961', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 02 09:47:18.031526 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 02 09:47:18.031526 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:47:18.031526 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 09:47:18.031526 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:47:18.031526 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:47:18.031526 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:47:18.031526 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:47:18.031526 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:47:18.038338 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:47:18.038338 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:47:18.038338 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:47:18.038338 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:47:18.038338 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:47:18.038338 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:47:18.038338 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:47:18.038338 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 02 09:47:18.038338 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 02 09:47:18.038338 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:47:18.038338 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:47:18.038338 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 09:47:18.038338 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 09:47:18.038338 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 09:47:18.038338 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 09:47:18.038338 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 02 09:47:18.038338 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 02 09:47:18.043070 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 02 09:47:18.043070 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 02 09:47:18.043070 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 02 09:47:18.043070 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 02 09:47:18.043070 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 02 09:47:18.043070 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:47:18.043070 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 473/958] 10.222.0.64 () {60 vars in 1131 bytes} [Tue Aug 2 09:47:17 2022] POST /identity/v3/users => generated 160 bytes in 252 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 02 09:47:18.050757 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7fe5c7-359b-4d09-88b8-68d1c45837ed None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:18.052412 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:18.099262 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:18.107310 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 486/959] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:47:17 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 02 09:47:18.131148 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a38a10-05f1-44d4-b817-d84db6c68c47 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:18.213303 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7fe5c7-359b-4d09-88b8-68d1c45837ed None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:18.255882 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3247be-564c-4df9-a931-136c342b420d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:18.256599 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3247be-564c-4df9-a931-136c342b420d None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:18.256989 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3247be-564c-4df9-a931-136c342b420d None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:18.257401 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3247be-564c-4df9-a931-136c342b420d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:18.267422 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3247be-564c-4df9-a931-136c342b420d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:18.269533 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3247be-564c-4df9-a931-136c342b420d None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:18.276892 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a38a10-05f1-44d4-b817-d84db6c68c47 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:18.285003 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-dc3247be-564c-4df9-a931-136c342b420d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 02 09:47:18.285003 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:47:18.285003 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:47:18.285003 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:47:18.285003 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:47:18.285003 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:47:18.285003 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:47:18.285003 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:47:18.285003 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:47:18.285003 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:47:18.285003 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:47:18.285003 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:47:18.285003 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 09:47:18.285003 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 09:47:18.285003 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 09:47:18.285003 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 09:47:18.286103 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:47:18.286103 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:47:18.286103 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 09:47:18.286103 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 09:47:18.286103 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 09:47:18.286103 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 09:47:18.286103 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:47:18.286103 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:47:18.286103 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:47:18.286103 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:47:18.286103 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:47:18.286103 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:47:18.286103 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:47:18.286103 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:47:18.286103 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:47:18.286103 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:47:18.286103 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:47:18.286103 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:47:18.287107 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:47:18.287107 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:47:18.287107 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 09:47:18.287107 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 09:47:18.287107 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 09:47:18.287107 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 09:47:18.287107 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 09:47:18.287107 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 09:47:18.287107 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 02 09:47:18.287107 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:47:18.287107 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 474/960] 10.222.0.64 () {56 vars in 1099 bytes} [Tue Aug 2 09:47:18 2022] GET /identity/v3/users/cinder => generated 84 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 09:47:18.293801 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3247be-564c-4df9-a931-136c342b420d None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:18.329716 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19590ed-ad59-4f3e-aa48-f865d61efb39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:18.331184 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19590ed-ad59-4f3e-aa48-f865d61efb39 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:18.331745 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19590ed-ad59-4f3e-aa48-f865d61efb39 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:18.332182 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19590ed-ad59-4f3e-aa48-f865d61efb39 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:18.333074 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19590ed-ad59-4f3e-aa48-f865d61efb39 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:18.335820 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19590ed-ad59-4f3e-aa48-f865d61efb39 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:18.349373 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-d19590ed-ad59-4f3e-aa48-f865d61efb39 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 02 09:47:18.349373 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:47:18.349373 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 09:47:18.349373 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 09:47:18.349373 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 09:47:18.349373 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 09:47:18.349373 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 09:47:18.349373 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 09:47:18.349373 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 02 09:47:18.349373 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:47:18.349373 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 09:47:18.349373 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:47:18.349373 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:47:18.349373 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:47:18.349373 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:47:18.349373 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:47:18.349373 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:47:18.350696 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:47:18.350696 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:47:18.350696 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:47:18.350696 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:47:18.350696 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:47:18.350696 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:47:18.350696 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 09:47:18.350696 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 09:47:18.350696 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 09:47:18.350696 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 09:47:18.350696 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:47:18.350696 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:47:18.350696 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:47:18.350696 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:47:18.350696 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:47:18.350696 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:47:18.351772 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:47:18.351772 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:47:18.351772 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:47:18.351772 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:47:18.351772 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:47:18.351772 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:47:18.351772 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:47:18.351772 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:47:18.351772 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:47:18.351772 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:47:18.351772 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 09:47:18.351772 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 09:47:18.351772 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 02 09:47:18.351772 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:47:18.355111 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 487/961] 10.222.0.64 () {56 vars in 1126 bytes} [Tue Aug 2 09:47:18 2022] GET /identity/v3/services/object-store => generated 93 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 09:47:18.364300 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19590ed-ad59-4f3e-aa48-f865d61efb39 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:18.396173 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:18.396755 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:18.416638 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3247be-564c-4df9-a931-136c342b420d None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:18.455204 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b11b851-43d9-4b51-a685-486ecb0328c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:18.456187 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b11b851-43d9-4b51-a685-486ecb0328c1 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:18.456749 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b11b851-43d9-4b51-a685-486ecb0328c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:18.457034 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b11b851-43d9-4b51-a685-486ecb0328c1 None admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:18.457799 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b11b851-43d9-4b51-a685-486ecb0328c1 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:47:18.458101 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b11b851-43d9-4b51-a685-486ecb0328c1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:18.459522 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b11b851-43d9-4b51-a685-486ecb0328c1 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:18.494568 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19590ed-ad59-4f3e-aa48-f865d61efb39 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:18.508279 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 475/962] 10.222.0.64 () {56 vars in 1101 bytes} [Tue Aug 2 09:47:18 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:47:18.538298 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c60b9e-7662-46e0-9e9c-8452d80a2fdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:18.538849 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c60b9e-7662-46e0-9e9c-8452d80a2fdd None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:18.539185 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c60b9e-7662-46e0-9e9c-8452d80a2fdd None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:18.539465 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c60b9e-7662-46e0-9e9c-8452d80a2fdd None admin] PATH_INFO: `/v3/services` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:18.540159 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c60b9e-7662-46e0-9e9c-8452d80a2fdd None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:47:18.540499 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c60b9e-7662-46e0-9e9c-8452d80a2fdd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:18.541983 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c60b9e-7662-46e0-9e9c-8452d80a2fdd None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:18.549245 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 488/963] 10.222.0.64 () {56 vars in 1122 bytes} [Tue Aug 2 09:47:18 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:47:18.555158 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b11b851-43d9-4b51-a685-486ecb0328c1 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:18.668652 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b11b851-43d9-4b51-a685-486ecb0328c1 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:18.718668 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07e386f-b8f8-48ea-b67e-4395ec997cd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:18.719661 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07e386f-b8f8-48ea-b67e-4395ec997cd4 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:18.720163 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07e386f-b8f8-48ea-b67e-4395ec997cd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:18.720477 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07e386f-b8f8-48ea-b67e-4395ec997cd4 None admin] PATH_INFO: `/v3/services` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:18.721177 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07e386f-b8f8-48ea-b67e-4395ec997cd4 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:47:18.721485 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07e386f-b8f8-48ea-b67e-4395ec997cd4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:18.722856 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07e386f-b8f8-48ea-b67e-4395ec997cd4 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:18.728996 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 476/964] 10.222.0.64 () {56 vars in 1122 bytes} [Tue Aug 2 09:47:18 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:47:18.735194 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c60b9e-7662-46e0-9e9c-8452d80a2fdd None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:18.835244 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:18.855362 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c60b9e-7662-46e0-9e9c-8452d80a2fdd None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:18.897417 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68fb0b8-c5d4-4cc4-ac11-60588c2edca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:18.898058 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68fb0b8-c5d4-4cc4-ac11-60588c2edca9 None admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:18.898308 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68fb0b8-c5d4-4cc4-ac11-60588c2edca9 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:18.898579 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68fb0b8-c5d4-4cc4-ac11-60588c2edca9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:18.899293 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68fb0b8-c5d4-4cc4-ac11-60588c2edca9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:18.900708 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68fb0b8-c5d4-4cc4-ac11-60588c2edca9 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:18.922596 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 489/965] 10.222.0.64 () {60 vars in 1144 bytes} [Tue Aug 2 09:47:18 2022] POST /identity/v3/endpoints => generated 349 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 09:47:19.606790 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9c27e2-ce50-4797-8bab-49add37c8688 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:19.607520 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9c27e2-ce50-4797-8bab-49add37c8688 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:19.607520 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9c27e2-ce50-4797-8bab-49add37c8688 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:19.609749 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 477/966] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:47:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:47:19.615285 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4427dab-dc53-4175-82f8-d697224371f3 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:19.615651 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4427dab-dc53-4175-82f8-d697224371f3 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:19.616001 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4427dab-dc53-4175-82f8-d697224371f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:19.638726 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-c4427dab-dc53-4175-82f8-d697224371f3 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:47:19.640241 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-c4427dab-dc53-4175-82f8-d697224371f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:47:19.742071 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 490/967] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:47:19 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 02 09:47:19.765197 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9c27e2-ce50-4797-8bab-49add37c8688 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:19.775100 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:19.898186 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9c27e2-ce50-4797-8bab-49add37c8688 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:19.938935 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c8400a-b732-49c2-86e3-bad7b0582bec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:19.939565 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c8400a-b732-49c2-86e3-bad7b0582bec None admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:19.939852 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c8400a-b732-49c2-86e3-bad7b0582bec None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:19.940127 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c8400a-b732-49c2-86e3-bad7b0582bec None admin] PATH_INFO: `/v3/services` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:19.940872 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c8400a-b732-49c2-86e3-bad7b0582bec None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:19.942263 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c8400a-b732-49c2-86e3-bad7b0582bec None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:19.958193 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 478/968] 10.222.0.64 () {60 vars in 1141 bytes} [Tue Aug 2 09:47:19 2022] POST /identity/v3/services => generated 242 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:47:20.167815 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb6611f-3adc-4ed3-a6ac-bec2991de62d None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:20.168355 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb6611f-3adc-4ed3-a6ac-bec2991de62d None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:20.168355 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb6611f-3adc-4ed3-a6ac-bec2991de62d None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:20.169815 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 491/969] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:47:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:47:20.175687 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a27a22-ee0b-4fc0-b455-3d4594338f20 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:20.175987 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a27a22-ee0b-4fc0-b455-3d4594338f20 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:20.176291 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a27a22-ee0b-4fc0-b455-3d4594338f20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:20.195016 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-f7a27a22-ee0b-4fc0-b455-3d4594338f20 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:47:20.196577 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-f7a27a22-ee0b-4fc0-b455-3d4594338f20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:47:20.260224 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:20.277311 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 479/970] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:47:20 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 09:47:20.300515 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb6611f-3adc-4ed3-a6ac-bec2991de62d None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:20.369680 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:20.407613 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb6611f-3adc-4ed3-a6ac-bec2991de62d None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:20.447300 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440e257a-676d-47aa-bfed-2cf6c729175e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:20.447869 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440e257a-676d-47aa-bfed-2cf6c729175e None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:20.448154 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440e257a-676d-47aa-bfed-2cf6c729175e None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:20.448462 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440e257a-676d-47aa-bfed-2cf6c729175e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:20.459822 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440e257a-676d-47aa-bfed-2cf6c729175e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 09:47:20.464811 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-440e257a-676d-47aa-bfed-2cf6c729175e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 02 09:47:20.464811 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:47:20.464811 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:47:20.464811 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:47:20.464811 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:47:20.464811 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:47:20.464811 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:47:20.464811 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:47:20.464811 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:47:20.464811 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:47:20.464811 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:47:20.464811 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:47:20.464811 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 09:47:20.464811 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 09:47:20.464811 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 09:47:20.464811 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 09:47:20.465860 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 09:47:20.465860 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 09:47:20.465860 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 09:47:20.465860 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 09:47:20.465860 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:47:20.465860 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:47:20.465860 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:47:20.465860 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:47:20.465860 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:47:20.465860 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:47:20.465860 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:47:20.465860 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:47:20.465860 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:47:20.465860 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:47:20.465860 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:47:20.465860 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:47:20.466734 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:47:20.466734 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:47:20.466734 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:47:20.466734 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:47:20.466734 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 09:47:20.466734 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 09:47:20.466734 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 09:47:20.466734 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 09:47:20.466734 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 09:47:20.466734 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 09:47:20.466734 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 02 09:47:20.466734 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:47:20.474912 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 492/971] 10.222.0.64 () {56 vars in 1120 bytes} [Tue Aug 2 09:47:20 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:47:20.475094 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a27a22-ee0b-4fc0-b455-3d4594338f20 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:20.583348 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a27a22-ee0b-4fc0-b455-3d4594338f20 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:20.622012 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d16eb8-7be6-4e2b-8a0b-1969f211c2b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:20.622656 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d16eb8-7be6-4e2b-8a0b-1969f211c2b3 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:20.622913 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d16eb8-7be6-4e2b-8a0b-1969f211c2b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:20.623189 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d16eb8-7be6-4e2b-8a0b-1969f211c2b3 None admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:20.624469 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d16eb8-7be6-4e2b-8a0b-1969f211c2b3 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:47:20.624782 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d16eb8-7be6-4e2b-8a0b-1969f211c2b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:20.626237 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d16eb8-7be6-4e2b-8a0b-1969f211c2b3 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:20.640157 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 480/972] 10.222.0.64 () {56 vars in 1115 bytes} [Tue Aug 2 09:47:20 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:47:20.641464 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440e257a-676d-47aa-bfed-2cf6c729175e None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:20.745130 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440e257a-676d-47aa-bfed-2cf6c729175e None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:20.772431 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:20.782261 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff843ad-b003-4b1b-91cb-e28515d625ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:20.782907 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff843ad-b003-4b1b-91cb-e28515d625ed None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:20.783194 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff843ad-b003-4b1b-91cb-e28515d625ed None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:20.783475 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff843ad-b003-4b1b-91cb-e28515d625ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:20.794351 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff843ad-b003-4b1b-91cb-e28515d625ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:20.795889 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff843ad-b003-4b1b-91cb-e28515d625ed None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:20.810573 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-6ff843ad-b003-4b1b-91cb-e28515d625ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:47:20.810573 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:47:20.810573 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:47:20.810573 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:47:20.810573 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:47:20.810573 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:47:20.810573 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:47:20.810573 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:47:20.810573 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:47:20.810573 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:47:20.810573 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:47:20.810573 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:47:20.810573 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 09:47:20.810573 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 09:47:20.810573 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 09:47:20.812030 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 09:47:20.812030 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:47:20.812030 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:47:20.812030 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:47:20.812030 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:47:20.812030 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:47:20.812030 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:47:20.812030 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:47:20.812030 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:47:20.812030 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:47:20.812030 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:47:20.812030 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:47:20.812030 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:47:20.812030 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:47:20.812030 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:47:20.812030 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:47:20.812030 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:47:20.817823 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 09:47:20.817823 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 09:47:20.817823 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:47:20.817823 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:47:20.817823 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 493/973] 10.222.0.64 () {56 vars in 1108 bytes} [Tue Aug 2 09:47:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:47:20.818132 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d16eb8-7be6-4e2b-8a0b-1969f211c2b3 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:20.890748 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:20.949794 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d16eb8-7be6-4e2b-8a0b-1969f211c2b3 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:20.990524 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ca8344-3894-4253-a5e7-5b9961a2b3cd None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:20.990953 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ca8344-3894-4253-a5e7-5b9961a2b3cd None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:20.991292 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ca8344-3894-4253-a5e7-5b9961a2b3cd None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:20.992929 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 494/974] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:47:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 09:47:20.999936 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8d129e-c359-4bcf-944c-5b26fb4806c6 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:21.000535 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8d129e-c359-4bcf-944c-5b26fb4806c6 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:21.000980 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8d129e-c359-4bcf-944c-5b26fb4806c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:21.005515 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f65160-4ba5-4ab9-b9b4-2f9dd02f0f59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:21.006431 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f65160-4ba5-4ab9-b9b4-2f9dd02f0f59 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:21.006925 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f65160-4ba5-4ab9-b9b4-2f9dd02f0f59 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:21.007413 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f65160-4ba5-4ab9-b9b4-2f9dd02f0f59 None admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:21.009034 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f65160-4ba5-4ab9-b9b4-2f9dd02f0f59 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:47:21.010891 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f65160-4ba5-4ab9-b9b4-2f9dd02f0f59 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:21.013385 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f65160-4ba5-4ab9-b9b4-2f9dd02f0f59 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:21.021199 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-ff8d129e-c359-4bcf-944c-5b26fb4806c6 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:47:21.022394 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-ff8d129e-c359-4bcf-944c-5b26fb4806c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:47:21.035452 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 481/975] 10.222.0.64 () {56 vars in 1109 bytes} [Tue Aug 2 09:47:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:47:21.041636 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f65160-4ba5-4ab9-b9b4-2f9dd02f0f59 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:21.125118 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 495/976] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:47:20 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 02 09:47:21.149519 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8d129e-c359-4bcf-944c-5b26fb4806c6 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:21.175362 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f65160-4ba5-4ab9-b9b4-2f9dd02f0f59 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:21.191022 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:21.218594 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a327f10-36f1-4527-99f3-c569b8dcfc3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:21.219615 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a327f10-36f1-4527-99f3-c569b8dcfc3a None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:21.220006 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a327f10-36f1-4527-99f3-c569b8dcfc3a None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:21.220485 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a327f10-36f1-4527-99f3-c569b8dcfc3a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:21.230908 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a327f10-36f1-4527-99f3-c569b8dcfc3a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:21.232405 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a327f10-36f1-4527-99f3-c569b8dcfc3a None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:21.245469 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-8a327f10-36f1-4527-99f3-c569b8dcfc3a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 02 09:47:21.245469 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:47:21.245469 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:47:21.245469 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:47:21.245469 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:47:21.245469 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:47:21.245469 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:47:21.245469 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:47:21.245469 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:47:21.245469 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:47:21.245469 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:47:21.245469 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:47:21.245469 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 09:47:21.245469 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 09:47:21.245469 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 09:47:21.245469 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 09:47:21.246586 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:47:21.246586 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:47:21.246586 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 09:47:21.246586 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 09:47:21.246586 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 09:47:21.246586 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 09:47:21.246586 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:47:21.246586 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:47:21.246586 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:47:21.246586 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:47:21.246586 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:47:21.246586 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:47:21.246586 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:47:21.246586 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:47:21.246586 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:47:21.246586 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:47:21.246586 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:47:21.246586 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:47:21.247563 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:47:21.247563 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:47:21.247563 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 09:47:21.247563 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 09:47:21.247563 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 09:47:21.247563 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 09:47:21.247563 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 09:47:21.247563 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 09:47:21.247563 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 02 09:47:21.247563 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:47:21.248657 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 482/977] 10.222.0.64 () {56 vars in 1117 bytes} [Tue Aug 2 09:47:21 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 09:47:21.254340 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a327f10-36f1-4527-99f3-c569b8dcfc3a None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:21.272146 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8d129e-c359-4bcf-944c-5b26fb4806c6 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:21.286439 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:21.314302 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e6042e-cc06-4de3-8936-735d66cffdee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:21.314948 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e6042e-cc06-4de3-8936-735d66cffdee None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:21.315230 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e6042e-cc06-4de3-8936-735d66cffdee None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:21.315506 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e6042e-cc06-4de3-8936-735d66cffdee None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:21.316146 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e6042e-cc06-4de3-8936-735d66cffdee None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:21.317574 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e6042e-cc06-4de3-8936-735d66cffdee None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:21.327756 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-83e6042e-cc06-4de3-8936-735d66cffdee None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 02 09:47:21.327756 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:47:21.327756 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 09:47:21.327756 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 09:47:21.327756 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 09:47:21.327756 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 09:47:21.327756 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 09:47:21.327756 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 09:47:21.327756 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 02 09:47:21.327756 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:47:21.327756 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 09:47:21.327756 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:47:21.327756 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:47:21.327756 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:47:21.327756 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:47:21.327756 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:47:21.327756 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:47:21.328955 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:47:21.328955 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:47:21.328955 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:47:21.328955 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:47:21.328955 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:47:21.328955 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:47:21.328955 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 09:47:21.328955 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 09:47:21.328955 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 09:47:21.328955 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 09:47:21.328955 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:47:21.328955 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:47:21.328955 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:47:21.328955 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:47:21.328955 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:47:21.328955 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:47:21.336172 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:47:21.336172 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:47:21.336172 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:47:21.336172 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:47:21.336172 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:47:21.336172 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:47:21.336172 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:47:21.336172 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:47:21.336172 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:47:21.336172 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:47:21.336172 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 09:47:21.336172 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 09:47:21.336172 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 02 09:47:21.336172 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:47:21.336172 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 496/978] 10.222.0.64 () {56 vars in 1126 bytes} [Tue Aug 2 09:47:21 2022] GET /identity/v3/services/object-store => generated 93 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 09:47:21.339616 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e6042e-cc06-4de3-8936-735d66cffdee None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:21.395637 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a327f10-36f1-4527-99f3-c569b8dcfc3a None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:21.435378 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42a3998-5542-4985-8a8d-3e74a8bd21a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:21.436091 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42a3998-5542-4985-8a8d-3e74a8bd21a6 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:21.437007 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42a3998-5542-4985-8a8d-3e74a8bd21a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:21.437352 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42a3998-5542-4985-8a8d-3e74a8bd21a6 None admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:21.438266 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42a3998-5542-4985-8a8d-3e74a8bd21a6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:47:21.438648 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42a3998-5542-4985-8a8d-3e74a8bd21a6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:21.440182 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42a3998-5542-4985-8a8d-3e74a8bd21a6 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:21.463961 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 483/979] 10.222.0.64 () {56 vars in 1149 bytes} [Tue Aug 2 09:47:21 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:47:21.465101 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42a3998-5542-4985-8a8d-3e74a8bd21a6 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:21.467813 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e6042e-cc06-4de3-8936-735d66cffdee None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:21.498106 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:21.506989 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba960f6-770d-4992-9214-6375ad8770c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:21.507538 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba960f6-770d-4992-9214-6375ad8770c0 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:21.507826 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba960f6-770d-4992-9214-6375ad8770c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:21.508108 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba960f6-770d-4992-9214-6375ad8770c0 None admin] PATH_INFO: `/v3/services` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:21.508850 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba960f6-770d-4992-9214-6375ad8770c0 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:47:21.509164 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba960f6-770d-4992-9214-6375ad8770c0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:21.510561 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba960f6-770d-4992-9214-6375ad8770c0 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:21.516701 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 497/980] 10.222.0.64 () {56 vars in 1122 bytes} [Tue Aug 2 09:47:21 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:47:21.522147 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba960f6-770d-4992-9214-6375ad8770c0 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:21.580105 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:21.581040 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42a3998-5542-4985-8a8d-3e74a8bd21a6 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:21.629802 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2a180d-bf2b-46f9-bd13-6bd51d3295d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:21.630462 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2a180d-bf2b-46f9-bd13-6bd51d3295d7 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:21.630899 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2a180d-bf2b-46f9-bd13-6bd51d3295d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:21.631123 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2a180d-bf2b-46f9-bd13-6bd51d3295d7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:21.650713 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba960f6-770d-4992-9214-6375ad8770c0 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:21.691422 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1319168-2247-47f2-a6d0-a278755ef925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:21.692048 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1319168-2247-47f2-a6d0-a278755ef925 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:21.692363 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1319168-2247-47f2-a6d0-a278755ef925 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:21.692639 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1319168-2247-47f2-a6d0-a278755ef925 None admin] PATH_INFO: `/v3/services` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:21.693348 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1319168-2247-47f2-a6d0-a278755ef925 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:47:21.693627 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1319168-2247-47f2-a6d0-a278755ef925 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:21.694994 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1319168-2247-47f2-a6d0-a278755ef925 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:21.705307 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 498/981] 10.222.0.64 () {56 vars in 1122 bytes} [Tue Aug 2 09:47:21 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:47:21.707053 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1319168-2247-47f2-a6d0-a278755ef925 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:21.726599 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2a180d-bf2b-46f9-bd13-6bd51d3295d7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:21.728171 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2a180d-bf2b-46f9-bd13-6bd51d3295d7 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:21.763908 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:21.830445 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1319168-2247-47f2-a6d0-a278755ef925 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:21.832215 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:21.861862 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 484/982] 10.222.0.64 () {58 vars in 1282 bytes} [Tue Aug 2 09:47:21 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 02 09:47:21.863134 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2a180d-bf2b-46f9-bd13-6bd51d3295d7 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:21.892341 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab557d33-eca2-4081-ae0a-d09a6758ee54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:21.893411 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab557d33-eca2-4081-ae0a-d09a6758ee54 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:21.893694 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab557d33-eca2-4081-ae0a-d09a6758ee54 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:21.894149 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab557d33-eca2-4081-ae0a-d09a6758ee54 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:21.894924 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab557d33-eca2-4081-ae0a-d09a6758ee54 None admin] RBAC: Adding query filter params (interface=admin, service_id=7a1ff592a4444f2880101f5890b14962, region_id=RegionOne) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:47:21.895234 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab557d33-eca2-4081-ae0a-d09a6758ee54 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:21.896828 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab557d33-eca2-4081-ae0a-d09a6758ee54 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:21.902595 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 499/983] 10.222.0.64 () {56 vars in 1249 bytes} [Tue Aug 2 09:47:21 2022] GET /identity/v3/endpoints?service_id=7a1ff592a4444f2880101f5890b14962&interface=admin®ion_id=RegionOne => generated 193 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:47:21.908112 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab557d33-eca2-4081-ae0a-d09a6758ee54 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:21.984847 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2a180d-bf2b-46f9-bd13-6bd51d3295d7 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:22.021657 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab557d33-eca2-4081-ae0a-d09a6758ee54 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:22.023403 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:22.029370 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6397ac-19fb-4c85-942d-182253b36e40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:22.029727 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6397ac-19fb-4c85-942d-182253b36e40 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:22.030044 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6397ac-19fb-4c85-942d-182253b36e40 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:22.030685 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6397ac-19fb-4c85-942d-182253b36e40 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:22.041535 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6397ac-19fb-4c85-942d-182253b36e40 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:22.043006 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6397ac-19fb-4c85-942d-182253b36e40 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:22.056791 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d86d447-97c3-435c-8ca8-da98c4235002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:22.057400 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d86d447-97c3-435c-8ca8-da98c4235002 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:22.057685 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d86d447-97c3-435c-8ca8-da98c4235002 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:22.057960 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d86d447-97c3-435c-8ca8-da98c4235002 None admin] PATH_INFO: `/v3/services` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:22.058610 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d86d447-97c3-435c-8ca8-da98c4235002 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:22.058795 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-cf6397ac-19fb-4c85-942d-182253b36e40 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:47:22.058795 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:47:22.058795 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:47:22.058795 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:47:22.058795 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:47:22.058795 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:47:22.058795 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:47:22.058795 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:47:22.058795 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:47:22.058795 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:47:22.058795 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:47:22.058795 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:47:22.058795 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 09:47:22.058795 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 09:47:22.058795 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 09:47:22.059817 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 09:47:22.059817 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:47:22.059817 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:47:22.059817 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:47:22.059817 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:47:22.059817 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:47:22.059817 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:47:22.059817 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:47:22.059817 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:47:22.059817 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:47:22.059817 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:47:22.059817 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:47:22.059817 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:47:22.059817 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:47:22.059817 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:47:22.059817 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:47:22.059817 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:47:22.060808 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 09:47:22.060808 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 09:47:22.060808 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:47:22.060808 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:47:22.061034 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d86d447-97c3-435c-8ca8-da98c4235002 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:22.061116 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 485/984] 10.222.0.64 () {56 vars in 1108 bytes} [Tue Aug 2 09:47:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:47:22.061880 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca6bb68-5d8e-4fd9-91b7-c2cef05a6910 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:22.062176 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca6bb68-5d8e-4fd9-91b7-c2cef05a6910 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:22.062454 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca6bb68-5d8e-4fd9-91b7-c2cef05a6910 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:22.063785 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 486/985] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:47:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:47:22.066131 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca6bb68-5d8e-4fd9-91b7-c2cef05a6910 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:22.077322 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 500/986] 10.222.0.64 () {56 vars in 1087 bytes} [Tue Aug 2 09:47:21 2022] GET /identity/v3/services => generated 1727 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:47:22.077322 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658e25be-685b-458a-baff-1faac48f89dc None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:22.077322 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658e25be-685b-458a-baff-1faac48f89dc None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:22.077322 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658e25be-685b-458a-baff-1faac48f89dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:22.092392 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-658e25be-685b-458a-baff-1faac48f89dc None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:47:22.094148 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-658e25be-685b-458a-baff-1faac48f89dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:47:22.122152 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:22.171277 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 501/987] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:47:22 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 09:47:22.194674 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658e25be-685b-458a-baff-1faac48f89dc None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:22.196867 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca6bb68-5d8e-4fd9-91b7-c2cef05a6910 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:22.238110 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce9aba8-ee86-41d4-ba9e-e52a8f94d68c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:22.239327 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce9aba8-ee86-41d4-ba9e-e52a8f94d68c None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:22.239618 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce9aba8-ee86-41d4-ba9e-e52a8f94d68c None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:22.240004 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce9aba8-ee86-41d4-ba9e-e52a8f94d68c None admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:22.241195 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce9aba8-ee86-41d4-ba9e-e52a8f94d68c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:47:22.243771 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce9aba8-ee86-41d4-ba9e-e52a8f94d68c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:22.243771 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce9aba8-ee86-41d4-ba9e-e52a8f94d68c None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:22.257601 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 487/988] 10.222.0.64 () {56 vars in 1109 bytes} [Tue Aug 2 09:47:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:47:22.263597 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce9aba8-ee86-41d4-ba9e-e52a8f94d68c None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:22.313205 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:22.329667 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658e25be-685b-458a-baff-1faac48f89dc None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:22.371984 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14910864-e028-4db4-92d6-61fee3fc9f85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:22.372477 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14910864-e028-4db4-92d6-61fee3fc9f85 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:22.372956 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14910864-e028-4db4-92d6-61fee3fc9f85 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:22.373426 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14910864-e028-4db4-92d6-61fee3fc9f85 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:22.374623 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14910864-e028-4db4-92d6-61fee3fc9f85 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:22.377258 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14910864-e028-4db4-92d6-61fee3fc9f85 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:22.390956 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce9aba8-ee86-41d4-ba9e-e52a8f94d68c None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:22.394198 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-14910864-e028-4db4-92d6-61fee3fc9f85 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 02 09:47:22.394198 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:47:22.394198 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 09:47:22.394198 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 09:47:22.394198 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 09:47:22.394198 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 09:47:22.394198 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 09:47:22.394198 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 09:47:22.394198 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 02 09:47:22.394198 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:47:22.394198 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 09:47:22.394198 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:47:22.394198 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:47:22.394198 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:47:22.394198 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:47:22.394198 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:47:22.394198 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:47:22.395249 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:47:22.395249 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:47:22.395249 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:47:22.395249 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:47:22.395249 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:47:22.395249 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:47:22.395249 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 09:47:22.395249 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 09:47:22.395249 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 09:47:22.395249 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 09:47:22.395249 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:47:22.395249 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:47:22.395249 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:47:22.395249 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:47:22.395249 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:47:22.395249 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:47:22.396126 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:47:22.396126 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:47:22.396126 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:47:22.396126 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:47:22.396126 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:47:22.396126 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:47:22.396126 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:47:22.396126 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:47:22.396126 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:47:22.396126 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:47:22.396126 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 09:47:22.396126 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 09:47:22.396126 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 02 09:47:22.396126 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:47:22.404562 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 502/989] 10.222.0.64 () {56 vars in 1111 bytes} [Tue Aug 2 09:47:22 2022] GET /identity/v3/services/compute => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 09:47:22.406901 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14910864-e028-4db4-92d6-61fee3fc9f85 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:22.428771 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29e32bb-4c91-415c-a299-b1e863e3e0ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:22.429520 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29e32bb-4c91-415c-a299-b1e863e3e0ee None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:22.429943 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29e32bb-4c91-415c-a299-b1e863e3e0ee None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:22.430409 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29e32bb-4c91-415c-a299-b1e863e3e0ee None admin] PATH_INFO: `/v3/projects/service` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:22.439460 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29e32bb-4c91-415c-a299-b1e863e3e0ee None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:22.441500 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29e32bb-4c91-415c-a299-b1e863e3e0ee None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:22.444410 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:22.456799 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-c29e32bb-4c91-415c-a299-b1e863e3e0ee None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 09:47:22.456799 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:47:22.456799 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:47:22.456799 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:47:22.456799 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:47:22.456799 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:47:22.456799 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:47:22.456799 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:47:22.456799 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:47:22.456799 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:47:22.456799 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:47:22.456799 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:47:22.456799 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 09:47:22.456799 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 09:47:22.456799 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 09:47:22.457874 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 09:47:22.457874 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:47:22.457874 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:47:22.457874 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:47:22.457874 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:47:22.457874 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:47:22.457874 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:47:22.457874 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:47:22.457874 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:47:22.457874 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:47:22.457874 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:47:22.457874 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:47:22.457874 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:47:22.457874 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:47:22.457874 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:47:22.457874 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:47:22.457874 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:47:22.459137 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 09:47:22.459137 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 09:47:22.459137 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 09:47:22.459137 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 09:47:22.459137 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 09:47:22.459137 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 09:47:22.459137 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 09:47:22.459137 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:47:22.459137 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 488/990] 10.222.0.64 () {56 vars in 1111 bytes} [Tue Aug 2 09:47:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 09:47:22.463589 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29e32bb-4c91-415c-a299-b1e863e3e0ee None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:22.532725 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14910864-e028-4db4-92d6-61fee3fc9f85 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:22.571888 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457a859f-ea64-446b-94c0-9fbb005ca0c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:22.572455 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457a859f-ea64-446b-94c0-9fbb005ca0c6 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:22.572738 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457a859f-ea64-446b-94c0-9fbb005ca0c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:22.573006 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457a859f-ea64-446b-94c0-9fbb005ca0c6 None admin] PATH_INFO: `/v3/services` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:22.573692 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457a859f-ea64-446b-94c0-9fbb005ca0c6 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:47:22.573998 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457a859f-ea64-446b-94c0-9fbb005ca0c6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:22.574174 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29e32bb-4c91-415c-a299-b1e863e3e0ee None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:22.575378 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457a859f-ea64-446b-94c0-9fbb005ca0c6 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:22.576703 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:22.582165 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 503/991] 10.222.0.64 () {56 vars in 1112 bytes} [Tue Aug 2 09:47:22 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:47:22.587946 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457a859f-ea64-446b-94c0-9fbb005ca0c6 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:22.613827 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98e0dff-19a0-41cd-bde2-c30eb01cccf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:22.614744 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98e0dff-19a0-41cd-bde2-c30eb01cccf2 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:22.615037 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98e0dff-19a0-41cd-bde2-c30eb01cccf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:22.615318 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98e0dff-19a0-41cd-bde2-c30eb01cccf2 None admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:22.616075 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98e0dff-19a0-41cd-bde2-c30eb01cccf2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:47:22.616429 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98e0dff-19a0-41cd-bde2-c30eb01cccf2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:22.617884 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98e0dff-19a0-41cd-bde2-c30eb01cccf2 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:22.630952 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 489/992] 10.222.0.64 () {56 vars in 1148 bytes} [Tue Aug 2 09:47:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:47:22.637492 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98e0dff-19a0-41cd-bde2-c30eb01cccf2 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:22.687820 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:22.703805 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457a859f-ea64-446b-94c0-9fbb005ca0c6 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:22.741188 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b09c305-7829-458a-a729-bb9e099f011d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:22.741815 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b09c305-7829-458a-a729-bb9e099f011d None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:22.742101 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b09c305-7829-458a-a729-bb9e099f011d None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:22.742378 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b09c305-7829-458a-a729-bb9e099f011d None admin] PATH_INFO: `/v3/services` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:22.743063 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b09c305-7829-458a-a729-bb9e099f011d None admin] RBAC: Adding query filter params (type=compute) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:47:22.743362 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b09c305-7829-458a-a729-bb9e099f011d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:22.744781 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b09c305-7829-458a-a729-bb9e099f011d None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:22.746806 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98e0dff-19a0-41cd-bde2-c30eb01cccf2 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:22.754619 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 504/993] 10.222.0.64 () {56 vars in 1112 bytes} [Tue Aug 2 09:47:22 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:47:22.756123 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b09c305-7829-458a-a729-bb9e099f011d None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:22.786867 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4c03f0-50ff-4031-850d-c1a3a26afd33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:22.787820 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4c03f0-50ff-4031-850d-c1a3a26afd33 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:22.788282 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4c03f0-50ff-4031-850d-c1a3a26afd33 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:22.788697 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4c03f0-50ff-4031-850d-c1a3a26afd33 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:22.789744 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4c03f0-50ff-4031-850d-c1a3a26afd33 None admin] RBAC: Adding query filter params (role.id=55b4c595b54f40c6b03ed7161e5258bf, scope.project.id=390bd7c4f91c4465b0c7ab0f9e436c4d, user.id=5338bc8c594e4bc9bc41019226bf8d87) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:47:22.790294 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4c03f0-50ff-4031-850d-c1a3a26afd33 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:22.791877 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4c03f0-50ff-4031-850d-c1a3a26afd33 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:22.808278 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 490/994] 10.222.0.64 () {56 vars in 1374 bytes} [Tue Aug 2 09:47:22 2022] GET /identity/v3/role_assignments?user.id=5338bc8c594e4bc9bc41019226bf8d87&scope.project.id=390bd7c4f91c4465b0c7ab0f9e436c4d&role.id=55b4c595b54f40c6b03ed7161e5258bf => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:47:22.886673 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b09c305-7829-458a-a729-bb9e099f011d None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:22.924917 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71407d8-d6c4-4c27-a9e5-53039c9cac7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:22.925555 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71407d8-d6c4-4c27-a9e5-53039c9cac7e None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:22.925835 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71407d8-d6c4-4c27-a9e5-53039c9cac7e None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:22.926103 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71407d8-d6c4-4c27-a9e5-53039c9cac7e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:22.926874 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71407d8-d6c4-4c27-a9e5-53039c9cac7e None admin] RBAC: Adding query filter params (interface=public, service_id=d9563155f8d94a9db49802c6b4d9708b, region_id=RegionOne) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:47:22.927186 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71407d8-d6c4-4c27-a9e5-53039c9cac7e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:22.928746 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71407d8-d6c4-4c27-a9e5-53039c9cac7e None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:22.939984 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 505/995] 10.222.0.64 () {56 vars in 1251 bytes} [Tue Aug 2 09:47:22 2022] GET /identity/v3/endpoints?service_id=d9563155f8d94a9db49802c6b4d9708b&interface=public®ion_id=RegionOne => generated 194 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:47:22.941047 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4c03f0-50ff-4031-850d-c1a3a26afd33 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:22.973050 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:23.051264 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4c03f0-50ff-4031-850d-c1a3a26afd33 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:23.087843 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac50767-9d2a-48eb-b3c3-ca6694e89df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:23.088498 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac50767-9d2a-48eb-b3c3-ca6694e89df2 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:23.088788 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac50767-9d2a-48eb-b3c3-ca6694e89df2 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:23.089062 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac50767-9d2a-48eb-b3c3-ca6694e89df2 None admin] PATH_INFO: `/v3/services` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:23.089722 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac50767-9d2a-48eb-b3c3-ca6694e89df2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:23.091098 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac50767-9d2a-48eb-b3c3-ca6694e89df2 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:23.104294 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 491/996] 10.222.0.64 () {56 vars in 1087 bytes} [Tue Aug 2 09:47:22 2022] GET /identity/v3/services => generated 1727 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:47:24.113370 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416e9694-5e79-4631-b53d-c0c5af7216bc None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:24.113899 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416e9694-5e79-4631-b53d-c0c5af7216bc None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:24.114022 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416e9694-5e79-4631-b53d-c0c5af7216bc None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:24.115454 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 506/997] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:47:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:47:24.120931 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abf1350-ef87-4014-bab2-007c0d28faeb None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:24.121392 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abf1350-ef87-4014-bab2-007c0d28faeb None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:24.121495 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abf1350-ef87-4014-bab2-007c0d28faeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:24.141195 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-5abf1350-ef87-4014-bab2-007c0d28faeb None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:47:24.142329 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-5abf1350-ef87-4014-bab2-007c0d28faeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:47:24.212518 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 492/998] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:47:24 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 02 09:47:24.235079 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416e9694-5e79-4631-b53d-c0c5af7216bc None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:24.256149 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:24.350277 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416e9694-5e79-4631-b53d-c0c5af7216bc None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:24.383348 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637dc186-d41a-4eee-aeea-8dbed00dd69b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:24.383983 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637dc186-d41a-4eee-aeea-8dbed00dd69b None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:24.384283 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637dc186-d41a-4eee-aeea-8dbed00dd69b None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:24.384560 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637dc186-d41a-4eee-aeea-8dbed00dd69b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:24.385184 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637dc186-d41a-4eee-aeea-8dbed00dd69b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:24.386543 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637dc186-d41a-4eee-aeea-8dbed00dd69b None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:24.395440 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-637dc186-d41a-4eee-aeea-8dbed00dd69b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 02 09:47:24.395440 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:47:24.395440 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 09:47:24.395440 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 09:47:24.395440 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 09:47:24.395440 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 09:47:24.395440 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 09:47:24.395440 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 09:47:24.395440 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 02 09:47:24.395440 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:47:24.395440 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 09:47:24.395440 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:47:24.395440 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:47:24.395440 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:47:24.395440 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:47:24.395440 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:47:24.395440 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:47:24.396701 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:47:24.396701 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:47:24.396701 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:47:24.396701 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:47:24.396701 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:47:24.396701 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:47:24.396701 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 09:47:24.396701 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 09:47:24.396701 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 09:47:24.396701 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 09:47:24.396701 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:47:24.396701 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:47:24.396701 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:47:24.396701 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:47:24.396701 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:47:24.396701 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:47:24.397615 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:47:24.397615 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:47:24.397615 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:47:24.397615 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:47:24.397615 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:47:24.397615 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:47:24.397615 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:47:24.397615 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:47:24.397615 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:47:24.397615 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:47:24.397615 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 09:47:24.397615 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 09:47:24.397615 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 02 09:47:24.397615 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:47:24.398871 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 507/999] 10.222.0.64 () {56 vars in 1126 bytes} [Tue Aug 2 09:47:24 2022] GET /identity/v3/services/object-store => generated 93 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:47:24.402671 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abf1350-ef87-4014-bab2-007c0d28faeb None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:24.427495 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:24.502800 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abf1350-ef87-4014-bab2-007c0d28faeb None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:24.536609 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b0b8c1-df09-404b-8462-f5eb06e96f19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:24.537225 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b0b8c1-df09-404b-8462-f5eb06e96f19 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:24.537512 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b0b8c1-df09-404b-8462-f5eb06e96f19 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:24.537786 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b0b8c1-df09-404b-8462-f5eb06e96f19 None admin] PATH_INFO: `/v3/services` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:24.538461 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b0b8c1-df09-404b-8462-f5eb06e96f19 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:47:24.538765 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b0b8c1-df09-404b-8462-f5eb06e96f19 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:24.540118 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b0b8c1-df09-404b-8462-f5eb06e96f19 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:24.545072 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 493/1000] 10.222.0.64 () {56 vars in 1122 bytes} [Tue Aug 2 09:47:24 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:47:24.551156 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637dc186-d41a-4eee-aeea-8dbed00dd69b None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:24.634043 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:24.655684 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637dc186-d41a-4eee-aeea-8dbed00dd69b None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:24.690028 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705a0935-658a-4385-a018-2f0bc603e445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:24.690644 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705a0935-658a-4385-a018-2f0bc603e445 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:24.690924 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705a0935-658a-4385-a018-2f0bc603e445 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:24.691195 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705a0935-658a-4385-a018-2f0bc603e445 None admin] PATH_INFO: `/v3/services` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:24.691925 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705a0935-658a-4385-a018-2f0bc603e445 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:47:24.692233 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705a0935-658a-4385-a018-2f0bc603e445 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:24.693702 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705a0935-658a-4385-a018-2f0bc603e445 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:24.702949 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 508/1001] 10.222.0.64 () {56 vars in 1122 bytes} [Tue Aug 2 09:47:24 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:47:24.704276 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b0b8c1-df09-404b-8462-f5eb06e96f19 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:24.793867 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:24.809824 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b0b8c1-df09-404b-8462-f5eb06e96f19 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:24.850174 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389b5377-af8f-4820-b30e-545f40766cee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:24.850817 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389b5377-af8f-4820-b30e-545f40766cee None admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:24.851102 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389b5377-af8f-4820-b30e-545f40766cee None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:24.851381 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389b5377-af8f-4820-b30e-545f40766cee None admin] PATH_INFO: `/v3/endpoints` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:24.852126 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389b5377-af8f-4820-b30e-545f40766cee None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:24.853540 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389b5377-af8f-4820-b30e-545f40766cee None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:24.868588 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bf7b15-ff60-4210-bff4-071271aa587b None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:24.869122 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bf7b15-ff60-4210-bff4-071271aa587b None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:24.869503 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bf7b15-ff60-4210-bff4-071271aa587b None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:24.871368 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 509/1002] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:47:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:47:24.872537 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 494/1003] 10.222.0.64 () {60 vars in 1144 bytes} [Tue Aug 2 09:47:24 2022] POST /identity/v3/endpoints => generated 325 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 09:47:24.877357 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661fd72c-3cef-4490-ad9c-dbdff8d435c9 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:24.877938 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661fd72c-3cef-4490-ad9c-dbdff8d435c9 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:24.878405 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661fd72c-3cef-4490-ad9c-dbdff8d435c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:24.900234 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-661fd72c-3cef-4490-ad9c-dbdff8d435c9 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:47:24.901536 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-661fd72c-3cef-4490-ad9c-dbdff8d435c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:47:25.006014 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 510/1004] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:47:24 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 09:47:25.032842 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389b5377-af8f-4820-b30e-545f40766cee None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:25.164381 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389b5377-af8f-4820-b30e-545f40766cee None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:25.172083 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667c6c1c-0184-40aa-a906-f8146632037a None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:25.172647 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667c6c1c-0184-40aa-a906-f8146632037a None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:25.172749 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667c6c1c-0184-40aa-a906-f8146632037a None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:25.174033 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 511/1005] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:47:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 09:47:25.179191 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599769d2-1063-4b7f-9d5a-2ff80e65668a None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:25.179486 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599769d2-1063-4b7f-9d5a-2ff80e65668a None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:25.179761 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599769d2-1063-4b7f-9d5a-2ff80e65668a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:25.199260 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-599769d2-1063-4b7f-9d5a-2ff80e65668a None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:47:25.200514 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-599769d2-1063-4b7f-9d5a-2ff80e65668a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:47:25.204515 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8124dab9-343f-41e0-846a-3d732b633cae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:25.205170 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8124dab9-343f-41e0-846a-3d732b633cae None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:25.205461 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8124dab9-343f-41e0-846a-3d732b633cae None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:25.205752 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8124dab9-343f-41e0-846a-3d732b633cae None admin] PATH_INFO: `/v3/services/image` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:25.206367 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8124dab9-343f-41e0-846a-3d732b633cae None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:25.207725 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8124dab9-343f-41e0-846a-3d732b633cae None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:25.216901 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-8124dab9-343f-41e0-846a-3d732b633cae None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 02 09:47:25.216901 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:47:25.216901 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 09:47:25.216901 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 09:47:25.216901 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 09:47:25.216901 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 09:47:25.216901 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 09:47:25.216901 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 09:47:25.216901 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 02 09:47:25.216901 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:47:25.216901 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 09:47:25.216901 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:47:25.216901 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:47:25.216901 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:47:25.216901 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:47:25.216901 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:47:25.216901 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:47:25.218097 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:47:25.218097 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:47:25.218097 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:47:25.218097 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:47:25.218097 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:47:25.218097 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:47:25.218097 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 09:47:25.218097 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 09:47:25.218097 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 09:47:25.218097 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 09:47:25.218097 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:47:25.218097 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:47:25.218097 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:47:25.218097 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:47:25.218097 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:47:25.218097 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:47:25.219048 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:47:25.219048 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:47:25.219048 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:47:25.219048 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:47:25.219048 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:47:25.219048 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:47:25.219048 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:47:25.219048 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:47:25.219048 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:47:25.219048 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:47:25.219048 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 09:47:25.219048 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 09:47:25.219048 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 02 09:47:25.219048 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:47:25.219048 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 495/1006] 10.222.0.64 () {56 vars in 1105 bytes} [Tue Aug 2 09:47:25 2022] GET /identity/v3/services/image => generated 86 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:47:25.224469 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8124dab9-343f-41e0-846a-3d732b633cae None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:25.237525 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:25.278753 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:25.285512 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 512/1007] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:47:25 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 02 09:47:25.307336 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599769d2-1063-4b7f-9d5a-2ff80e65668a None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:25.342963 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8124dab9-343f-41e0-846a-3d732b633cae None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:25.386129 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9139b8-667f-4e9a-af46-121c297a5121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:25.386777 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9139b8-667f-4e9a-af46-121c297a5121 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:25.387071 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9139b8-667f-4e9a-af46-121c297a5121 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:25.387340 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9139b8-667f-4e9a-af46-121c297a5121 None admin] PATH_INFO: `/v3/services` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:25.388027 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9139b8-667f-4e9a-af46-121c297a5121 None admin] RBAC: Adding query filter params (name=image) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:47:25.388387 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9139b8-667f-4e9a-af46-121c297a5121 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:25.389803 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9139b8-667f-4e9a-af46-121c297a5121 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:25.400436 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 496/1008] 10.222.0.64 () {56 vars in 1108 bytes} [Tue Aug 2 09:47:25 2022] GET /identity/v3/services?name=image => generated 122 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:47:25.401549 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9139b8-667f-4e9a-af46-121c297a5121 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:25.422372 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599769d2-1063-4b7f-9d5a-2ff80e65668a None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:25.458430 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eba42e-bb9c-410d-bc3e-dc7d0d00c18a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:25.459082 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eba42e-bb9c-410d-bc3e-dc7d0d00c18a None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:25.459371 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eba42e-bb9c-410d-bc3e-dc7d0d00c18a None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:25.459649 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eba42e-bb9c-410d-bc3e-dc7d0d00c18a None admin] PATH_INFO: `/v3/services/compute` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:25.460311 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15eba42e-bb9c-410d-bc3e-dc7d0d00c18a None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:25.461694 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15eba42e-bb9c-410d-bc3e-dc7d0d00c18a None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:25.471659 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-15eba42e-bb9c-410d-bc3e-dc7d0d00c18a None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 02 09:47:25.471659 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:47:25.471659 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 09:47:25.471659 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 09:47:25.471659 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 09:47:25.471659 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 09:47:25.471659 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 09:47:25.471659 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 09:47:25.471659 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 02 09:47:25.471659 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:47:25.471659 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 09:47:25.471659 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:47:25.471659 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:47:25.471659 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:47:25.471659 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:47:25.471659 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:47:25.471659 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:47:25.472924 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:47:25.472924 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:47:25.472924 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:47:25.472924 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:47:25.472924 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:47:25.472924 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:47:25.472924 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 09:47:25.472924 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 09:47:25.472924 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 09:47:25.472924 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 09:47:25.472924 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:47:25.472924 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:47:25.472924 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:47:25.472924 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:47:25.472924 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:47:25.472924 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:47:25.474309 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:47:25.474309 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:47:25.474309 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:47:25.474309 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:47:25.474309 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:47:25.474309 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:47:25.474309 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:47:25.474309 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:47:25.474309 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:47:25.474309 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:47:25.474309 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 09:47:25.474309 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 09:47:25.474309 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 02 09:47:25.474309 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:47:25.474309 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 513/1009] 10.222.0.64 () {56 vars in 1111 bytes} [Tue Aug 2 09:47:25 2022] GET /identity/v3/services/compute => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:47:25.478889 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eba42e-bb9c-410d-bc3e-dc7d0d00c18a None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:25.487919 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:25.517286 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9139b8-667f-4e9a-af46-121c297a5121 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:25.542890 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:25.557172 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3c17d6-570a-4dea-9fde-2d8dd0b4de4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:25.557652 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3c17d6-570a-4dea-9fde-2d8dd0b4de4f None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:25.557934 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3c17d6-570a-4dea-9fde-2d8dd0b4de4f None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:25.558207 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3c17d6-570a-4dea-9fde-2d8dd0b4de4f None admin] PATH_INFO: `/v3/services` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:25.558892 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3c17d6-570a-4dea-9fde-2d8dd0b4de4f None admin] RBAC: Adding query filter params (type=image) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:47:25.559398 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3c17d6-570a-4dea-9fde-2d8dd0b4de4f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:25.560597 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3c17d6-570a-4dea-9fde-2d8dd0b4de4f None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:25.572361 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 497/1010] 10.222.0.64 () {56 vars in 1108 bytes} [Tue Aug 2 09:47:25 2022] GET /identity/v3/services?type=image => generated 122 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:47:25.593654 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eba42e-bb9c-410d-bc3e-dc7d0d00c18a None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:25.633215 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125d6e5a-96da-40e2-8b11-f53cbfa1cae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:25.633880 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125d6e5a-96da-40e2-8b11-f53cbfa1cae8 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:25.634165 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125d6e5a-96da-40e2-8b11-f53cbfa1cae8 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:25.634437 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125d6e5a-96da-40e2-8b11-f53cbfa1cae8 None admin] PATH_INFO: `/v3/services` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:25.635136 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125d6e5a-96da-40e2-8b11-f53cbfa1cae8 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:47:25.635441 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125d6e5a-96da-40e2-8b11-f53cbfa1cae8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:25.636889 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125d6e5a-96da-40e2-8b11-f53cbfa1cae8 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:25.647642 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 514/1011] 10.222.0.64 () {56 vars in 1112 bytes} [Tue Aug 2 09:47:25 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:47:25.649198 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3c17d6-570a-4dea-9fde-2d8dd0b4de4f None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:25.760086 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3c17d6-570a-4dea-9fde-2d8dd0b4de4f None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:25.794069 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa8b153-9382-4860-8394-ff1689af63e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:25.794608 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa8b153-9382-4860-8394-ff1689af63e9 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:25.794888 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa8b153-9382-4860-8394-ff1689af63e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:25.795152 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa8b153-9382-4860-8394-ff1689af63e9 None admin] PATH_INFO: `/v3/services` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:25.796477 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa8b153-9382-4860-8394-ff1689af63e9 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:47:25.796786 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa8b153-9382-4860-8394-ff1689af63e9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:25.798141 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa8b153-9382-4860-8394-ff1689af63e9 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:25.807698 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 498/1012] 10.222.0.64 () {56 vars in 1112 bytes} [Tue Aug 2 09:47:25 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:47:25.808951 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125d6e5a-96da-40e2-8b11-f53cbfa1cae8 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:25.931005 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125d6e5a-96da-40e2-8b11-f53cbfa1cae8 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:25.959281 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:25.976271 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105c6d59-a9ea-412a-bcee-21181a59376f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:25.976271 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105c6d59-a9ea-412a-bcee-21181a59376f None admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:25.976271 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105c6d59-a9ea-412a-bcee-21181a59376f None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:25.976271 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105c6d59-a9ea-412a-bcee-21181a59376f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:25.976682 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105c6d59-a9ea-412a-bcee-21181a59376f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:25.980274 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105c6d59-a9ea-412a-bcee-21181a59376f None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:26.012276 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 515/1013] 10.222.0.64 () {60 vars in 1144 bytes} [Tue Aug 2 09:47:25 2022] POST /identity/v3/endpoints => generated 334 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:47:26.966327 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a064a4e-b8ac-4334-bfcb-d1ce9ba920cd None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:26.966876 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a064a4e-b8ac-4334-bfcb-d1ce9ba920cd None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:26.966876 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a064a4e-b8ac-4334-bfcb-d1ce9ba920cd None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:26.969402 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 499/1014] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:47:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:47:26.975008 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0735b606-cb3a-47e6-bcb7-deba4c2c8c11 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:26.975467 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0735b606-cb3a-47e6-bcb7-deba4c2c8c11 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:26.975543 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0735b606-cb3a-47e6-bcb7-deba4c2c8c11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:26.994916 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-0735b606-cb3a-47e6-bcb7-deba4c2c8c11 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:47:26.996091 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-0735b606-cb3a-47e6-bcb7-deba4c2c8c11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:47:27.071501 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 516/1015] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:47:26 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 09:47:27.094474 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a064a4e-b8ac-4334-bfcb-d1ce9ba920cd None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:27.097393 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:27.196601 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a064a4e-b8ac-4334-bfcb-d1ce9ba920cd None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:27.231548 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2b8c9d-6d14-42ee-8203-4456eb128296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:27.232191 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2b8c9d-6d14-42ee-8203-4456eb128296 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:27.232505 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2b8c9d-6d14-42ee-8203-4456eb128296 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:27.232881 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2b8c9d-6d14-42ee-8203-4456eb128296 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:27.234051 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2b8c9d-6d14-42ee-8203-4456eb128296 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:27.235503 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2b8c9d-6d14-42ee-8203-4456eb128296 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:27.237392 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 500/1016] 10.222.0.64 () {56 vars in 1108 bytes} [Tue Aug 2 09:47:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:47:27.243283 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0735b606-cb3a-47e6-bcb7-deba4c2c8c11 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:27.351874 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0735b606-cb3a-47e6-bcb7-deba4c2c8c11 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:27.363331 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:27.388537 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fe7c6c-1747-467f-a19a-4062b002eb2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:27.389212 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fe7c6c-1747-467f-a19a-4062b002eb2f None admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:27.390040 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fe7c6c-1747-467f-a19a-4062b002eb2f None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:27.390259 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fe7c6c-1747-467f-a19a-4062b002eb2f None admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:27.390934 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fe7c6c-1747-467f-a19a-4062b002eb2f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:27.392390 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fe7c6c-1747-467f-a19a-4062b002eb2f None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:27.427689 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 517/1017] 10.222.0.64 () {60 vars in 1141 bytes} [Tue Aug 2 09:47:27 2022] POST /identity/v3/projects => generated 311 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:47:27.625556 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42902d71-47db-4946-a72a-5071e24f2c62 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:27.625886 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42902d71-47db-4946-a72a-5071e24f2c62 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:27.626125 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42902d71-47db-4946-a72a-5071e24f2c62 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:27.627626 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 501/1018] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:47:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:47:27.632802 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102e984e-587e-41da-9bc9-5e568f493cdd None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:27.633100 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102e984e-587e-41da-9bc9-5e568f493cdd None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:27.633385 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102e984e-587e-41da-9bc9-5e568f493cdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:27.651510 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-102e984e-587e-41da-9bc9-5e568f493cdd None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:47:27.652731 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-102e984e-587e-41da-9bc9-5e568f493cdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:47:27.724346 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 518/1019] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:47:27 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 09:47:27.743501 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42902d71-47db-4946-a72a-5071e24f2c62 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:27.815082 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:27.854347 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42902d71-47db-4946-a72a-5071e24f2c62 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:27.894750 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8df48a-8405-44bd-80d7-b2d23e85617d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:27.895378 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8df48a-8405-44bd-80d7-b2d23e85617d None admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:27.895664 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8df48a-8405-44bd-80d7-b2d23e85617d None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:27.896002 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8df48a-8405-44bd-80d7-b2d23e85617d None admin] PATH_INFO: `/v3/services` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:27.896769 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8df48a-8405-44bd-80d7-b2d23e85617d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:27.898181 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8df48a-8405-44bd-80d7-b2d23e85617d None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:27.919344 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 502/1020] 10.222.0.64 () {60 vars in 1141 bytes} [Tue Aug 2 09:47:27 2022] POST /identity/v3/services => generated 242 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:47:28.162772 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8004a7-6a68-4222-b3b5-61341fe6e5d9 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:28.163619 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8004a7-6a68-4222-b3b5-61341fe6e5d9 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:28.163619 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8004a7-6a68-4222-b3b5-61341fe6e5d9 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:28.164997 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 519/1021] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:47:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:47:28.174166 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839fabf3-e6e3-4c57-a6c9-4404d672a2c7 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:28.174808 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839fabf3-e6e3-4c57-a6c9-4404d672a2c7 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:28.175289 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839fabf3-e6e3-4c57-a6c9-4404d672a2c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:28.196085 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-839fabf3-e6e3-4c57-a6c9-4404d672a2c7 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:47:28.197407 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-839fabf3-e6e3-4c57-a6c9-4404d672a2c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:47:28.275890 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 503/1022] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:47:28 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 09:47:28.298770 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8004a7-6a68-4222-b3b5-61341fe6e5d9 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:28.401972 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:28.413934 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8004a7-6a68-4222-b3b5-61341fe6e5d9 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:28.450585 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5525cec0-4cdc-4e63-81c6-eb27c4439c1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:28.451230 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5525cec0-4cdc-4e63-81c6-eb27c4439c1c None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:28.451620 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5525cec0-4cdc-4e63-81c6-eb27c4439c1c None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:28.452429 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5525cec0-4cdc-4e63-81c6-eb27c4439c1c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:28.462930 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5525cec0-4cdc-4e63-81c6-eb27c4439c1c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 09:47:28.467861 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-5525cec0-4cdc-4e63-81c6-eb27c4439c1c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 02 09:47:28.467861 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:47:28.467861 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:47:28.467861 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:47:28.467861 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:47:28.467861 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:47:28.467861 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:47:28.467861 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:47:28.467861 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:47:28.467861 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:47:28.467861 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:47:28.467861 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:47:28.467861 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 09:47:28.467861 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 09:47:28.467861 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 09:47:28.467861 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 09:47:28.469007 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 09:47:28.469007 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 09:47:28.469007 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 09:47:28.469007 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 09:47:28.469007 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:47:28.469007 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:47:28.469007 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:47:28.469007 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:47:28.469007 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:47:28.469007 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:47:28.469007 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:47:28.469007 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:47:28.469007 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:47:28.469007 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:47:28.469007 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:47:28.469007 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:47:28.469876 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:47:28.469876 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:47:28.469876 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:47:28.469876 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:47:28.469876 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 09:47:28.469876 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 09:47:28.469876 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 09:47:28.469876 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 09:47:28.469876 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 09:47:28.469876 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 09:47:28.469876 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 02 09:47:28.469876 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:47:28.472332 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 520/1023] 10.222.0.64 () {56 vars in 1120 bytes} [Tue Aug 2 09:47:28 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:47:28.475729 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839fabf3-e6e3-4c57-a6c9-4404d672a2c7 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:28.534951 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:28.583017 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839fabf3-e6e3-4c57-a6c9-4404d672a2c7 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:28.618473 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c6dde0-a14f-413f-a714-e95b267c051f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:28.619095 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c6dde0-a14f-413f-a714-e95b267c051f None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:28.619375 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c6dde0-a14f-413f-a714-e95b267c051f None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:28.619645 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c6dde0-a14f-413f-a714-e95b267c051f None admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:28.620390 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c6dde0-a14f-413f-a714-e95b267c051f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:47:28.620694 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c6dde0-a14f-413f-a714-e95b267c051f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:28.622054 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c6dde0-a14f-413f-a714-e95b267c051f None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:28.632281 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 504/1024] 10.222.0.64 () {56 vars in 1115 bytes} [Tue Aug 2 09:47:28 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:47:28.637028 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5525cec0-4cdc-4e63-81c6-eb27c4439c1c None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:28.742078 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5525cec0-4cdc-4e63-81c6-eb27c4439c1c None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:28.778149 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313547e5-903e-440a-b388-cf6144e6930b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:28.778791 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313547e5-903e-440a-b388-cf6144e6930b None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:28.779073 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313547e5-903e-440a-b388-cf6144e6930b None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:28.779348 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313547e5-903e-440a-b388-cf6144e6930b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:28.790496 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313547e5-903e-440a-b388-cf6144e6930b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:28.791956 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313547e5-903e-440a-b388-cf6144e6930b None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:28.807216 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-313547e5-903e-440a-b388-cf6144e6930b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:47:28.807216 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:47:28.807216 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:47:28.807216 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:47:28.807216 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:47:28.807216 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:47:28.807216 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:47:28.807216 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:47:28.807216 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:47:28.807216 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:47:28.807216 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:47:28.807216 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:47:28.807216 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 09:47:28.807216 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 09:47:28.807216 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 09:47:28.808206 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 09:47:28.808206 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:47:28.808206 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:47:28.808206 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:47:28.808206 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:47:28.808206 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:47:28.808206 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:47:28.808206 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:47:28.808206 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:47:28.808206 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:47:28.808206 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:47:28.808206 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:47:28.808206 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:47:28.808206 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:47:28.808206 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:47:28.808206 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:47:28.808206 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:47:28.809176 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 09:47:28.809176 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 09:47:28.809176 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:47:28.809176 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:47:28.809176 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 521/1025] 10.222.0.64 () {56 vars in 1108 bytes} [Tue Aug 2 09:47:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:47:28.816149 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c6dde0-a14f-413f-a714-e95b267c051f None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:28.932027 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c6dde0-a14f-413f-a714-e95b267c051f None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:28.950286 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:28.969701 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84640a91-3521-42fc-a2d1-a698c70dde53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:28.970330 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84640a91-3521-42fc-a2d1-a698c70dde53 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:28.970618 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84640a91-3521-42fc-a2d1-a698c70dde53 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:28.970881 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84640a91-3521-42fc-a2d1-a698c70dde53 None admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:28.971583 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84640a91-3521-42fc-a2d1-a698c70dde53 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:47:28.971885 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84640a91-3521-42fc-a2d1-a698c70dde53 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:28.973284 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84640a91-3521-42fc-a2d1-a698c70dde53 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:28.987099 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 505/1026] 10.222.0.64 () {56 vars in 1109 bytes} [Tue Aug 2 09:47:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:47:28.992539 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313547e5-903e-440a-b388-cf6144e6930b None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:28.997260 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:29.095235 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313547e5-903e-440a-b388-cf6144e6930b None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:29.131934 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f80cbe-59c5-47f8-bbc1-6a5e1c75e456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:29.132629 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f80cbe-59c5-47f8-bbc1-6a5e1c75e456 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:29.132914 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f80cbe-59c5-47f8-bbc1-6a5e1c75e456 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:29.133190 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f80cbe-59c5-47f8-bbc1-6a5e1c75e456 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:29.142332 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f80cbe-59c5-47f8-bbc1-6a5e1c75e456 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:29.143833 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f80cbe-59c5-47f8-bbc1-6a5e1c75e456 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:29.157110 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-96f80cbe-59c5-47f8-bbc1-6a5e1c75e456 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 02 09:47:29.157110 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:47:29.157110 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:47:29.157110 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:47:29.157110 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:47:29.157110 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:47:29.157110 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:47:29.157110 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:47:29.157110 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:47:29.157110 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:47:29.157110 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:47:29.157110 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:47:29.157110 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 09:47:29.157110 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 09:47:29.157110 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 09:47:29.157110 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 09:47:29.158184 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:47:29.158184 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:47:29.158184 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 09:47:29.158184 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 09:47:29.158184 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 09:47:29.158184 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 09:47:29.158184 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:47:29.158184 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:47:29.158184 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:47:29.158184 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:47:29.158184 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:47:29.158184 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:47:29.158184 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:47:29.158184 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:47:29.158184 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:47:29.158184 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:47:29.158184 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:47:29.158184 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:47:29.159215 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:47:29.159215 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:47:29.159215 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 09:47:29.159215 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 09:47:29.159215 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 09:47:29.159215 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 09:47:29.159215 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 09:47:29.159215 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 09:47:29.159215 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 02 09:47:29.159215 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:47:29.159215 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 522/1027] 10.222.0.64 () {56 vars in 1093 bytes} [Tue Aug 2 09:47:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:47:29.164123 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84640a91-3521-42fc-a2d1-a698c70dde53 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:29.269235 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84640a91-3521-42fc-a2d1-a698c70dde53 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:29.303295 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ffbc97-d738-4dad-aab5-0575c0b32c5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:29.303906 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ffbc97-d738-4dad-aab5-0575c0b32c5a None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:29.304186 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ffbc97-d738-4dad-aab5-0575c0b32c5a None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:29.304487 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ffbc97-d738-4dad-aab5-0575c0b32c5a None admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:29.305279 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ffbc97-d738-4dad-aab5-0575c0b32c5a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:47:29.305584 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ffbc97-d738-4dad-aab5-0575c0b32c5a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:29.307011 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ffbc97-d738-4dad-aab5-0575c0b32c5a None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:29.328327 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 506/1028] 10.222.0.64 () {56 vars in 1133 bytes} [Tue Aug 2 09:47:29 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:47:29.330822 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f80cbe-59c5-47f8-bbc1-6a5e1c75e456 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:29.405685 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:29.451064 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f80cbe-59c5-47f8-bbc1-6a5e1c75e456 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:29.484740 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d5653b-7487-47cd-aeb2-f4d368735035 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:29.485059 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d5653b-7487-47cd-aeb2-f4d368735035 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:29.485303 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d5653b-7487-47cd-aeb2-f4d368735035 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:29.486882 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 507/1029] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:47:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:47:29.492289 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33abf7f8-9786-4c97-8cbf-372764d35eb0 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:29.492585 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33abf7f8-9786-4c97-8cbf-372764d35eb0 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:29.492843 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33abf7f8-9786-4c97-8cbf-372764d35eb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:29.495680 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f18da62-bc1f-4ce9-bcc6-7e5bd4b0359c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:29.495680 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f18da62-bc1f-4ce9-bcc6-7e5bd4b0359c None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:29.495680 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f18da62-bc1f-4ce9-bcc6-7e5bd4b0359c None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:29.495965 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f18da62-bc1f-4ce9-bcc6-7e5bd4b0359c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:29.512460 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-33abf7f8-9786-4c97-8cbf-372764d35eb0 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:47:29.513749 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-33abf7f8-9786-4c97-8cbf-372764d35eb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:47:29.548319 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:29.572734 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f18da62-bc1f-4ce9-bcc6-7e5bd4b0359c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:29.574204 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f18da62-bc1f-4ce9-bcc6-7e5bd4b0359c None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:29.594887 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 508/1030] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:47:29 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 02 09:47:29.618889 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33abf7f8-9786-4c97-8cbf-372764d35eb0 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:29.643345 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:29.684761 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 523/1031] 10.222.0.64 () {58 vars in 1282 bytes} [Tue Aug 2 09:47:29 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 02 09:47:29.692147 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f18da62-bc1f-4ce9-bcc6-7e5bd4b0359c None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:29.730918 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33abf7f8-9786-4c97-8cbf-372764d35eb0 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:29.769602 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42d3827-b0ad-4266-8361-1a29c6f5a614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:29.770914 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42d3827-b0ad-4266-8361-1a29c6f5a614 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:29.770914 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42d3827-b0ad-4266-8361-1a29c6f5a614 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:29.770914 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42d3827-b0ad-4266-8361-1a29c6f5a614 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:29.771926 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42d3827-b0ad-4266-8361-1a29c6f5a614 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:29.773404 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42d3827-b0ad-4266-8361-1a29c6f5a614 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:29.775201 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 509/1032] 10.222.0.64 () {56 vars in 1108 bytes} [Tue Aug 2 09:47:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:47:29.780869 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42d3827-b0ad-4266-8361-1a29c6f5a614 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:29.801578 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f18da62-bc1f-4ce9-bcc6-7e5bd4b0359c None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:29.814787 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:29.841227 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4735ff-4748-4821-a146-a06c6406df55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:29.841865 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4735ff-4748-4821-a146-a06c6406df55 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:29.842179 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4735ff-4748-4821-a146-a06c6406df55 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:29.842493 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4735ff-4748-4821-a146-a06c6406df55 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:29.856208 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4735ff-4748-4821-a146-a06c6406df55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:29.857707 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4735ff-4748-4821-a146-a06c6406df55 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:29.872590 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-5f4735ff-4748-4821-a146-a06c6406df55 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:47:29.872590 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:47:29.872590 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:47:29.872590 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:47:29.872590 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:47:29.872590 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:47:29.872590 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:47:29.872590 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:47:29.872590 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:47:29.872590 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:47:29.872590 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:47:29.872590 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:47:29.872590 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 09:47:29.872590 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 09:47:29.872590 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 09:47:29.874081 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 09:47:29.874081 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:47:29.874081 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:47:29.874081 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:47:29.874081 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:47:29.874081 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:47:29.874081 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:47:29.874081 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:47:29.874081 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:47:29.874081 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:47:29.874081 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:47:29.874081 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:47:29.874081 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:47:29.874081 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:47:29.874081 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:47:29.874081 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:47:29.874081 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:47:29.880302 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 09:47:29.880302 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 09:47:29.880302 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:47:29.880302 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:47:29.880302 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 524/1033] 10.222.0.64 () {56 vars in 1108 bytes} [Tue Aug 2 09:47:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 09:47:29.880302 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4735ff-4748-4821-a146-a06c6406df55 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:29.895250 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42d3827-b0ad-4266-8361-1a29c6f5a614 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:29.902116 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:29.934151 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58eebd62-d663-42a6-a1ef-bc5ba5c89bc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:29.934836 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eebd62-d663-42a6-a1ef-bc5ba5c89bc8 None admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:29.935192 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eebd62-d663-42a6-a1ef-bc5ba5c89bc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:29.935576 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eebd62-d663-42a6-a1ef-bc5ba5c89bc8 None admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:29.936849 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58eebd62-d663-42a6-a1ef-bc5ba5c89bc8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:29.938284 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58eebd62-d663-42a6-a1ef-bc5ba5c89bc8 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:29.963439 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 510/1034] 10.222.0.64 () {60 vars in 1132 bytes} [Tue Aug 2 09:47:29 2022] POST /identity/v3/users => generated 285 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 09:47:30.008176 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4735ff-4748-4821-a146-a06c6406df55 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:30.057543 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed433693-221a-42e7-9ebd-becef208ed46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:30.058188 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed433693-221a-42e7-9ebd-becef208ed46 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:30.058475 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed433693-221a-42e7-9ebd-becef208ed46 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:30.058758 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed433693-221a-42e7-9ebd-becef208ed46 None admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:30.059480 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed433693-221a-42e7-9ebd-becef208ed46 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:47:30.060073 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed433693-221a-42e7-9ebd-becef208ed46 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:30.061845 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed433693-221a-42e7-9ebd-becef208ed46 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:30.072850 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f6fd8f-a861-4f38-bc08-9d5123c4a887 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:30.073213 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f6fd8f-a861-4f38-bc08-9d5123c4a887 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:30.073426 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f6fd8f-a861-4f38-bc08-9d5123c4a887 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:30.074995 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 511/1035] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:47:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 09:47:30.076075 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 525/1036] 10.222.0.64 () {56 vars in 1109 bytes} [Tue Aug 2 09:47:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:47:30.080480 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9208ca35-64c4-4872-8a7c-836dcaf7ffd2 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:30.080892 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9208ca35-64c4-4872-8a7c-836dcaf7ffd2 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:30.081274 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9208ca35-64c4-4872-8a7c-836dcaf7ffd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:30.081389 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed433693-221a-42e7-9ebd-becef208ed46 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:30.106250 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-9208ca35-64c4-4872-8a7c-836dcaf7ffd2 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:47:30.108298 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-9208ca35-64c4-4872-8a7c-836dcaf7ffd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:47:30.186752 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:30.218364 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed433693-221a-42e7-9ebd-becef208ed46 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:30.220491 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 512/1037] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:47:30 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 02 09:47:30.244666 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9208ca35-64c4-4872-8a7c-836dcaf7ffd2 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:30.263883 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3fe171-df1a-4c77-91b2-d5cd00b282cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:30.265097 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3fe171-df1a-4c77-91b2-d5cd00b282cb None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:30.265352 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3fe171-df1a-4c77-91b2-d5cd00b282cb None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:30.265626 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3fe171-df1a-4c77-91b2-d5cd00b282cb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:30.272055 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:30.273323 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3fe171-df1a-4c77-91b2-d5cd00b282cb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:30.275152 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3fe171-df1a-4c77-91b2-d5cd00b282cb None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:30.287848 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-fc3fe171-df1a-4c77-91b2-d5cd00b282cb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 09:47:30.287848 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:47:30.287848 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:47:30.287848 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:47:30.287848 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:47:30.287848 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:47:30.287848 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:47:30.287848 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:47:30.287848 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:47:30.287848 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:47:30.287848 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:47:30.287848 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:47:30.287848 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 09:47:30.287848 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 09:47:30.287848 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 09:47:30.288888 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 09:47:30.288888 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:47:30.288888 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:47:30.288888 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:47:30.288888 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:47:30.288888 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:47:30.288888 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:47:30.288888 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:47:30.288888 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:47:30.288888 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:47:30.288888 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:47:30.288888 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:47:30.288888 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:47:30.288888 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:47:30.288888 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:47:30.288888 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:47:30.288888 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:47:30.295022 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 09:47:30.295022 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 09:47:30.295022 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 09:47:30.295022 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 09:47:30.295022 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 09:47:30.295022 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 09:47:30.295022 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 09:47:30.295022 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:47:30.295022 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 526/1038] 10.222.0.64 () {56 vars in 1111 bytes} [Tue Aug 2 09:47:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 09:47:30.295623 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3fe171-df1a-4c77-91b2-d5cd00b282cb None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:30.366226 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9208ca35-64c4-4872-8a7c-836dcaf7ffd2 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:30.402480 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb801e4-0b31-4132-bfde-a0b938d88579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:30.403152 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb801e4-0b31-4132-bfde-a0b938d88579 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:30.403432 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb801e4-0b31-4132-bfde-a0b938d88579 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:30.403710 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb801e4-0b31-4132-bfde-a0b938d88579 None admin] PATH_INFO: `/v3/services/image` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:30.404371 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb801e4-0b31-4132-bfde-a0b938d88579 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:30.405953 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb801e4-0b31-4132-bfde-a0b938d88579 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:30.410480 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3fe171-df1a-4c77-91b2-d5cd00b282cb None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:30.415274 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-9bb801e4-0b31-4132-bfde-a0b938d88579 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 02 09:47:30.415274 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:47:30.415274 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 09:47:30.415274 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 09:47:30.415274 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 09:47:30.415274 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 09:47:30.415274 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 09:47:30.415274 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 09:47:30.415274 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 02 09:47:30.415274 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:47:30.415274 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 09:47:30.415274 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:47:30.415274 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:47:30.415274 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:47:30.415274 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:47:30.415274 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:47:30.415274 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:47:30.416342 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:47:30.416342 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:47:30.416342 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:47:30.416342 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:47:30.416342 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:47:30.416342 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:47:30.416342 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 09:47:30.416342 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 09:47:30.416342 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 09:47:30.416342 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 09:47:30.416342 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:47:30.416342 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:47:30.416342 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:47:30.416342 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:47:30.416342 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:47:30.416342 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:47:30.417225 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:47:30.417225 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:47:30.417225 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:47:30.417225 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:47:30.417225 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:47:30.417225 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:47:30.417225 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:47:30.417225 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:47:30.417225 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:47:30.417225 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:47:30.417225 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 09:47:30.417225 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 09:47:30.417225 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 02 09:47:30.417225 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:47:30.421617 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 513/1039] 10.222.0.64 () {56 vars in 1105 bytes} [Tue Aug 2 09:47:30 2022] GET /identity/v3/services/image => generated 86 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 09:47:30.422297 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb801e4-0b31-4132-bfde-a0b938d88579 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:30.459210 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2400bb9e-3c60-41fd-8c17-79e1408ca1a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:30.460236 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2400bb9e-3c60-41fd-8c17-79e1408ca1a8 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:30.460768 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2400bb9e-3c60-41fd-8c17-79e1408ca1a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:30.461208 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2400bb9e-3c60-41fd-8c17-79e1408ca1a8 None admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:30.462534 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2400bb9e-3c60-41fd-8c17-79e1408ca1a8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:47:30.463022 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2400bb9e-3c60-41fd-8c17-79e1408ca1a8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:30.465470 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2400bb9e-3c60-41fd-8c17-79e1408ca1a8 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:30.468612 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:30.481756 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 527/1040] 10.222.0.64 () {56 vars in 1148 bytes} [Tue Aug 2 09:47:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:47:30.488426 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2400bb9e-3c60-41fd-8c17-79e1408ca1a8 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:30.523706 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:30.543577 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb801e4-0b31-4132-bfde-a0b938d88579 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:30.587029 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9a6ca5-39bc-460b-a5aa-78d8352a518f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:30.587656 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9a6ca5-39bc-460b-a5aa-78d8352a518f None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:30.588055 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9a6ca5-39bc-460b-a5aa-78d8352a518f None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:30.588206 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9a6ca5-39bc-460b-a5aa-78d8352a518f None admin] PATH_INFO: `/v3/services` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:30.588917 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9a6ca5-39bc-460b-a5aa-78d8352a518f None admin] RBAC: Adding query filter params (name=image) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:47:30.589222 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9a6ca5-39bc-460b-a5aa-78d8352a518f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:30.590603 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9a6ca5-39bc-460b-a5aa-78d8352a518f None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:30.601470 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 514/1041] 10.222.0.64 () {56 vars in 1108 bytes} [Tue Aug 2 09:47:30 2022] GET /identity/v3/services?name=image => generated 122 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:47:30.602666 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9a6ca5-39bc-460b-a5aa-78d8352a518f None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:30.619063 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2400bb9e-3c60-41fd-8c17-79e1408ca1a8 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:30.656659 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d242464a-d5f1-4ce5-b142-ddce8490b875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:30.657279 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d242464a-d5f1-4ce5-b142-ddce8490b875 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:30.657572 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d242464a-d5f1-4ce5-b142-ddce8490b875 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:30.657847 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d242464a-d5f1-4ce5-b142-ddce8490b875 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:30.658646 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d242464a-d5f1-4ce5-b142-ddce8490b875 None admin] RBAC: Adding query filter params (role.id=55b4c595b54f40c6b03ed7161e5258bf, scope.project.id=390bd7c4f91c4465b0c7ab0f9e436c4d, user.id=99655e1f9ca24472a86cd26b0dbd1fc2) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:47:30.658958 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d242464a-d5f1-4ce5-b142-ddce8490b875 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:30.660427 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d242464a-d5f1-4ce5-b142-ddce8490b875 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:30.666026 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 528/1042] 10.222.0.64 () {56 vars in 1374 bytes} [Tue Aug 2 09:47:30 2022] GET /identity/v3/role_assignments?user.id=99655e1f9ca24472a86cd26b0dbd1fc2&scope.project.id=390bd7c4f91c4465b0c7ab0f9e436c4d&role.id=55b4c595b54f40c6b03ed7161e5258bf => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:47:30.713337 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9a6ca5-39bc-460b-a5aa-78d8352a518f None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:30.751823 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f20672-ac0c-435a-ac82-4c0de345503b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:30.752487 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f20672-ac0c-435a-ac82-4c0de345503b None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:30.752775 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f20672-ac0c-435a-ac82-4c0de345503b None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:30.753050 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f20672-ac0c-435a-ac82-4c0de345503b None admin] PATH_INFO: `/v3/services` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:30.753759 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f20672-ac0c-435a-ac82-4c0de345503b None admin] RBAC: Adding query filter params (type=image) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:47:30.754066 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f20672-ac0c-435a-ac82-4c0de345503b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:30.755487 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f20672-ac0c-435a-ac82-4c0de345503b None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:30.761256 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 515/1043] 10.222.0.64 () {56 vars in 1108 bytes} [Tue Aug 2 09:47:30 2022] GET /identity/v3/services?type=image => generated 350 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:47:30.767718 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d242464a-d5f1-4ce5-b142-ddce8490b875 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:30.837907 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:30.882006 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d242464a-d5f1-4ce5-b142-ddce8490b875 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:30.920629 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e402c167-ec71-4f8b-b010-44e16f6219b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:30.921250 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e402c167-ec71-4f8b-b010-44e16f6219b6 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:30.921537 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e402c167-ec71-4f8b-b010-44e16f6219b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:30.921813 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e402c167-ec71-4f8b-b010-44e16f6219b6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:30.922569 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e402c167-ec71-4f8b-b010-44e16f6219b6 None admin] RBAC: Adding query filter params (interface=public, service_id=e1d982c6f36f4ea2afe46aee7264d5ba, region_id=RegionOne) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:47:30.922885 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e402c167-ec71-4f8b-b010-44e16f6219b6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:30.924279 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e402c167-ec71-4f8b-b010-44e16f6219b6 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:30.934128 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 529/1044] 10.222.0.64 () {56 vars in 1251 bytes} [Tue Aug 2 09:47:30 2022] GET /identity/v3/endpoints?service_id=e1d982c6f36f4ea2afe46aee7264d5ba&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:47:30.935434 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f20672-ac0c-435a-ac82-4c0de345503b None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:30.951361 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:31.046686 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f20672-ac0c-435a-ac82-4c0de345503b None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:31.080944 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d62325-c5a3-484d-be1c-2c1abfa3dd8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:31.081562 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d62325-c5a3-484d-be1c-2c1abfa3dd8e None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:31.081845 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d62325-c5a3-484d-be1c-2c1abfa3dd8e None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:31.082172 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d62325-c5a3-484d-be1c-2c1abfa3dd8e None admin] PATH_INFO: `/v3/services` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:31.082828 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d62325-c5a3-484d-be1c-2c1abfa3dd8e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:31.084265 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d62325-c5a3-484d-be1c-2c1abfa3dd8e None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:31.096350 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 516/1045] 10.222.0.64 () {56 vars in 1087 bytes} [Tue Aug 2 09:47:30 2022] GET /identity/v3/services => generated 1957 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:47:32.098219 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9a7157-9005-4a1b-a423-aeadff7f8a68 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:32.099041 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9a7157-9005-4a1b-a423-aeadff7f8a68 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:32.099041 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9a7157-9005-4a1b-a423-aeadff7f8a68 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:32.104137 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 530/1046] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:47:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:47:32.105739 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dda145a-4e9c-4ad2-b508-159bc56f7c57 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:32.106137 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dda145a-4e9c-4ad2-b508-159bc56f7c57 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:32.106295 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dda145a-4e9c-4ad2-b508-159bc56f7c57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:32.124955 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-1dda145a-4e9c-4ad2-b508-159bc56f7c57 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:47:32.126146 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-1dda145a-4e9c-4ad2-b508-159bc56f7c57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:47:32.208375 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 517/1047] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:47:32 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 09:47:32.228757 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9a7157-9005-4a1b-a423-aeadff7f8a68 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:32.331856 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9a7157-9005-4a1b-a423-aeadff7f8a68 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:32.356698 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:32.368668 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5130c233-f78f-4a0f-be39-9492c045964a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:32.369541 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5130c233-f78f-4a0f-be39-9492c045964a None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:32.370002 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5130c233-f78f-4a0f-be39-9492c045964a None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:32.370452 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5130c233-f78f-4a0f-be39-9492c045964a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:32.380584 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5130c233-f78f-4a0f-be39-9492c045964a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 09:47:32.385600 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-5130c233-f78f-4a0f-be39-9492c045964a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 09:47:32.385600 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:47:32.385600 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:47:32.385600 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:47:32.385600 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:47:32.385600 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:47:32.385600 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:47:32.385600 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:47:32.385600 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:47:32.385600 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:47:32.385600 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:47:32.385600 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:47:32.385600 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 09:47:32.385600 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 09:47:32.385600 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 09:47:32.385600 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 09:47:32.386515 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 09:47:32.386515 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 09:47:32.386515 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 09:47:32.386515 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 09:47:32.386515 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:47:32.386515 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:47:32.386515 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:47:32.386515 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:47:32.386515 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:47:32.386515 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:47:32.386515 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:47:32.386515 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:47:32.386515 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:47:32.386515 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:47:32.386515 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:47:32.386515 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:47:32.387381 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:47:32.387381 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:47:32.387381 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:47:32.387381 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:47:32.387381 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 09:47:32.387381 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 09:47:32.387381 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 09:47:32.387381 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 09:47:32.387381 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 09:47:32.387381 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 09:47:32.387381 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 09:47:32.387381 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:47:32.389995 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 531/1048] 10.222.0.64 () {56 vars in 1096 bytes} [Tue Aug 2 09:47:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:47:32.395821 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dda145a-4e9c-4ad2-b508-159bc56f7c57 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:32.404383 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:32.499571 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dda145a-4e9c-4ad2-b508-159bc56f7c57 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:32.533925 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa83fed1-5012-4a89-94c6-d6152900737d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:32.534541 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa83fed1-5012-4a89-94c6-d6152900737d None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:32.534824 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa83fed1-5012-4a89-94c6-d6152900737d None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:32.535094 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa83fed1-5012-4a89-94c6-d6152900737d None admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:32.535810 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa83fed1-5012-4a89-94c6-d6152900737d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:47:32.536111 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa83fed1-5012-4a89-94c6-d6152900737d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:32.537501 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa83fed1-5012-4a89-94c6-d6152900737d None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:32.550121 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 518/1049] 10.222.0.64 () {56 vars in 1099 bytes} [Tue Aug 2 09:47:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:47:32.551966 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5130c233-f78f-4a0f-be39-9492c045964a None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:32.654744 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5130c233-f78f-4a0f-be39-9492c045964a None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:32.688654 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d96697-9f65-4164-bb5e-a1985c419f33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:32.689290 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d96697-9f65-4164-bb5e-a1985c419f33 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:32.689571 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d96697-9f65-4164-bb5e-a1985c419f33 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:32.689845 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d96697-9f65-4164-bb5e-a1985c419f33 None admin] PATH_INFO: `/v3/users/9e98f79dd6114048a6a4683e6e0c8809` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:32.703662 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d96697-9f65-4164-bb5e-a1985c419f33 None admin] RBAC: Authorizing `identity:get_user(user_id=9e98f79dd6114048a6a4683e6e0c8809)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:32.705142 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d96697-9f65-4164-bb5e-a1985c419f33 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:32.711422 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 532/1050] 10.222.0.64 () {56 vars in 1177 bytes} [Tue Aug 2 09:47:32 2022] GET /identity/v3/users/9e98f79dd6114048a6a4683e6e0c8809 => generated 285 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:47:32.712769 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa83fed1-5012-4a89-94c6-d6152900737d None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:32.790015 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:32.833824 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa83fed1-5012-4a89-94c6-d6152900737d None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:32.879349 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351d3c22-6e89-435d-9af5-1e4d89b5c971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:32.880653 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351d3c22-6e89-435d-9af5-1e4d89b5c971 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:32.881300 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351d3c22-6e89-435d-9af5-1e4d89b5c971 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:32.881968 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351d3c22-6e89-435d-9af5-1e4d89b5c971 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:32.916875 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c32516-591d-4dab-8272-5d6fa9e971b1 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:32.917169 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c32516-591d-4dab-8272-5d6fa9e971b1 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:32.917455 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c32516-591d-4dab-8272-5d6fa9e971b1 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:32.920311 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 533/1051] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:47:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:47:32.924095 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7555c4-3c09-4317-ba3f-8b245ba04516 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:32.924478 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7555c4-3c09-4317-ba3f-8b245ba04516 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:32.924707 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7555c4-3c09-4317-ba3f-8b245ba04516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:32.945061 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-be7555c4-3c09-4317-ba3f-8b245ba04516 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:47:32.946520 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-be7555c4-3c09-4317-ba3f-8b245ba04516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:47:32.969270 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351d3c22-6e89-435d-9af5-1e4d89b5c971 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:32.971321 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351d3c22-6e89-435d-9af5-1e4d89b5c971 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:32.990581 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:33.025796 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 534/1052] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:47:32 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 02 09:47:33.044414 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:33.049935 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7555c4-3c09-4317-ba3f-8b245ba04516 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:33.094198 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 519/1053] 10.222.0.64 () {58 vars in 1282 bytes} [Tue Aug 2 09:47:32 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 02 09:47:33.096605 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351d3c22-6e89-435d-9af5-1e4d89b5c971 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:33.156603 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7555c4-3c09-4317-ba3f-8b245ba04516 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:33.193593 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d8dd6d-fc31-4857-ae09-2a9eec308b90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:33.194245 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d8dd6d-fc31-4857-ae09-2a9eec308b90 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:33.194528 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d8dd6d-fc31-4857-ae09-2a9eec308b90 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:33.194800 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d8dd6d-fc31-4857-ae09-2a9eec308b90 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:33.200785 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351d3c22-6e89-435d-9af5-1e4d89b5c971 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:33.206311 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d8dd6d-fc31-4857-ae09-2a9eec308b90 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 09:47:33.212394 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-94d8dd6d-fc31-4857-ae09-2a9eec308b90 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 02 09:47:33.212394 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:47:33.212394 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:47:33.212394 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:47:33.212394 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:47:33.212394 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:47:33.212394 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:47:33.212394 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:47:33.212394 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:47:33.212394 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:47:33.212394 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:47:33.212394 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:47:33.212394 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 09:47:33.212394 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 09:47:33.212394 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 09:47:33.212394 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 09:47:33.213373 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 09:47:33.213373 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 09:47:33.213373 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 09:47:33.213373 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 09:47:33.213373 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:47:33.213373 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:47:33.213373 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:47:33.213373 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:47:33.213373 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:47:33.213373 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:47:33.213373 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:47:33.213373 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:47:33.213373 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:47:33.213373 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:47:33.213373 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:47:33.213373 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:47:33.214254 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:47:33.214254 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:47:33.214254 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:47:33.214254 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:47:33.214254 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 09:47:33.214254 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 09:47:33.214254 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 09:47:33.214254 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 09:47:33.214254 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 09:47:33.214254 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 09:47:33.214254 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 02 09:47:33.214254 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:47:33.214254 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 535/1054] 10.222.0.64 () {56 vars in 1120 bytes} [Tue Aug 2 09:47:33 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 09:47:33.214997 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eba27b-474d-4630-a080-671f23b890bc None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:33.215083 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eba27b-474d-4630-a080-671f23b890bc None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:33.215314 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eba27b-474d-4630-a080-671f23b890bc None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:33.217977 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 536/1055] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:47:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:47:33.219503 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52eba27b-474d-4630-a080-671f23b890bc None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:33.239237 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c98b50b-c13c-4c86-9966-82544314eec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:33.239875 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c98b50b-c13c-4c86-9966-82544314eec1 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:33.240154 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c98b50b-c13c-4c86-9966-82544314eec1 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:33.240449 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c98b50b-c13c-4c86-9966-82544314eec1 None admin] PATH_INFO: `/v3/projects/e502fb4fefa64be9b2e40bdec3c99876` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:33.241606 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c98b50b-c13c-4c86-9966-82544314eec1 None admin] RBAC: Authorizing `identity:get_project(project_id=e502fb4fefa64be9b2e40bdec3c99876)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:33.243038 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c98b50b-c13c-4c86-9966-82544314eec1 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:33.244748 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 520/1056] 10.222.0.64 () {56 vars in 1186 bytes} [Tue Aug 2 09:47:33 2022] GET /identity/v3/projects/e502fb4fefa64be9b2e40bdec3c99876 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:47:33.245873 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44b7257-b88b-4e66-bf8a-0815bb1b2491 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:33.246157 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44b7257-b88b-4e66-bf8a-0815bb1b2491 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:33.246430 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44b7257-b88b-4e66-bf8a-0815bb1b2491 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:33.253982 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:33.264881 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-a44b7257-b88b-4e66-bf8a-0815bb1b2491 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:47:33.266081 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-a44b7257-b88b-4e66-bf8a-0815bb1b2491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:47:33.295849 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:33.328547 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52eba27b-474d-4630-a080-671f23b890bc None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:33.335306 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 521/1057] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:47:33 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 09:47:33.336407 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44b7257-b88b-4e66-bf8a-0815bb1b2491 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:33.366645 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50564f3b-d794-404a-9e3d-d68bd478bfb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:33.367265 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50564f3b-d794-404a-9e3d-d68bd478bfb4 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:33.367557 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50564f3b-d794-404a-9e3d-d68bd478bfb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:33.367842 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50564f3b-d794-404a-9e3d-d68bd478bfb4 None admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:33.368609 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50564f3b-d794-404a-9e3d-d68bd478bfb4 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:47:33.368913 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50564f3b-d794-404a-9e3d-d68bd478bfb4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:33.370303 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50564f3b-d794-404a-9e3d-d68bd478bfb4 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:33.379332 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 537/1058] 10.222.0.64 () {56 vars in 1115 bytes} [Tue Aug 2 09:47:33 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:47:33.380541 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50564f3b-d794-404a-9e3d-d68bd478bfb4 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:33.444894 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44b7257-b88b-4e66-bf8a-0815bb1b2491 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:33.472869 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:33.479604 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba94009-5793-4a7c-b847-b99f3be23b1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:33.480238 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba94009-5793-4a7c-b847-b99f3be23b1c None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:33.480552 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba94009-5793-4a7c-b847-b99f3be23b1c None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:33.480820 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50564f3b-d794-404a-9e3d-d68bd478bfb4 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:33.480929 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba94009-5793-4a7c-b847-b99f3be23b1c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:33.481636 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba94009-5793-4a7c-b847-b99f3be23b1c None admin] RBAC: Adding query filter params (role.id=1000cf318fa14915b8091599eaead5de, scope.project.id=e502fb4fefa64be9b2e40bdec3c99876, user.id=9e98f79dd6114048a6a4683e6e0c8809) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:47:33.481952 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba94009-5793-4a7c-b847-b99f3be23b1c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:33.483549 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba94009-5793-4a7c-b847-b99f3be23b1c None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:33.490328 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 522/1059] 10.222.0.64 () {56 vars in 1374 bytes} [Tue Aug 2 09:47:33 2022] GET /identity/v3/role_assignments?user.id=9e98f79dd6114048a6a4683e6e0c8809&scope.project.id=e502fb4fefa64be9b2e40bdec3c99876&role.id=1000cf318fa14915b8091599eaead5de => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:47:33.491775 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba94009-5793-4a7c-b847-b99f3be23b1c None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:33.534901 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9be283-b504-4923-9a5e-76eee6117524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:33.535558 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9be283-b504-4923-9a5e-76eee6117524 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:33.535868 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9be283-b504-4923-9a5e-76eee6117524 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:33.536111 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9be283-b504-4923-9a5e-76eee6117524 None admin] PATH_INFO: `/v3/services/image` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:33.536930 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9be283-b504-4923-9a5e-76eee6117524 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:33.537418 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:33.538364 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9be283-b504-4923-9a5e-76eee6117524 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:33.550191 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-bd9be283-b504-4923-9a5e-76eee6117524 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 02 09:47:33.550191 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:47:33.550191 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 09:47:33.550191 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 09:47:33.550191 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 09:47:33.550191 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 09:47:33.550191 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 09:47:33.550191 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 09:47:33.550191 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 02 09:47:33.550191 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:47:33.550191 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 09:47:33.550191 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:47:33.550191 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:47:33.550191 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:47:33.550191 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:47:33.550191 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:47:33.550191 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:47:33.551304 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:47:33.551304 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:47:33.551304 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:47:33.551304 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:47:33.551304 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:47:33.551304 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:47:33.551304 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 09:47:33.551304 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 09:47:33.551304 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 09:47:33.551304 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 09:47:33.551304 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:47:33.551304 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:47:33.551304 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:47:33.551304 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:47:33.551304 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:47:33.551304 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:47:33.552225 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:47:33.552225 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:47:33.552225 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:47:33.552225 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:47:33.552225 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:47:33.552225 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:47:33.552225 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:47:33.552225 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:47:33.552225 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:47:33.552225 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:47:33.552225 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 09:47:33.552225 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 09:47:33.552225 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 02 09:47:33.552225 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:47:33.552225 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 538/1060] 10.222.0.64 () {56 vars in 1105 bytes} [Tue Aug 2 09:47:33 2022] GET /identity/v3/services/image => generated 86 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:47:33.557663 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9be283-b504-4923-9a5e-76eee6117524 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:33.627171 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba94009-5793-4a7c-b847-b99f3be23b1c None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:33.674527 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c449c295-0ac1-49d8-ac72-0c51ce773b69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:33.675273 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c449c295-0ac1-49d8-ac72-0c51ce773b69 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:33.675629 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c449c295-0ac1-49d8-ac72-0c51ce773b69 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:33.675976 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c449c295-0ac1-49d8-ac72-0c51ce773b69 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:33.676941 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9be283-b504-4923-9a5e-76eee6117524 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:33.687912 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c449c295-0ac1-49d8-ac72-0c51ce773b69 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:33.689410 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c449c295-0ac1-49d8-ac72-0c51ce773b69 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:33.704563 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-c449c295-0ac1-49d8-ac72-0c51ce773b69 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:47:33.704563 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:47:33.704563 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:47:33.704563 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:47:33.704563 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:47:33.704563 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:47:33.704563 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:47:33.704563 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:47:33.704563 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:47:33.704563 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:47:33.704563 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:47:33.704563 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:47:33.704563 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 09:47:33.704563 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 09:47:33.704563 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 09:47:33.705554 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 09:47:33.705554 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:47:33.705554 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:47:33.705554 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:47:33.705554 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:47:33.705554 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:47:33.705554 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:47:33.705554 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:47:33.705554 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:47:33.705554 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:47:33.705554 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:47:33.705554 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:47:33.705554 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:47:33.705554 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:47:33.705554 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:47:33.705554 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:47:33.705554 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:47:33.706504 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 09:47:33.706504 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 09:47:33.706504 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:47:33.706504 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:47:33.706504 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 523/1061] 10.222.0.64 () {56 vars in 1108 bytes} [Tue Aug 2 09:47:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:47:33.711309 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c449c295-0ac1-49d8-ac72-0c51ce773b69 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:33.718497 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ab0c3b-3f7d-42c3-80a1-c1ebdb916b3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:33.719124 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ab0c3b-3f7d-42c3-80a1-c1ebdb916b3b None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:33.719412 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ab0c3b-3f7d-42c3-80a1-c1ebdb916b3b None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:33.719690 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ab0c3b-3f7d-42c3-80a1-c1ebdb916b3b None admin] PATH_INFO: `/v3/services` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:33.720836 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ab0c3b-3f7d-42c3-80a1-c1ebdb916b3b None admin] RBAC: Adding query filter params (name=image) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:47:33.721154 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ab0c3b-3f7d-42c3-80a1-c1ebdb916b3b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:33.722906 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ab0c3b-3f7d-42c3-80a1-c1ebdb916b3b None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:33.730029 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 539/1062] 10.222.0.64 () {56 vars in 1108 bytes} [Tue Aug 2 09:47:33 2022] GET /identity/v3/services?name=image => generated 122 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:47:33.735861 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ab0c3b-3f7d-42c3-80a1-c1ebdb916b3b None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:33.761740 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:33.803167 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:33.823131 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c449c295-0ac1-49d8-ac72-0c51ce773b69 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:33.852763 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ab0c3b-3f7d-42c3-80a1-c1ebdb916b3b None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:33.859755 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d29d59-af09-4187-a866-b357284242fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:33.860418 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d29d59-af09-4187-a866-b357284242fb None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:33.860709 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d29d59-af09-4187-a866-b357284242fb None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:33.860988 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d29d59-af09-4187-a866-b357284242fb None admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:33.861717 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d29d59-af09-4187-a866-b357284242fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:47:33.862024 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d29d59-af09-4187-a866-b357284242fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:33.863394 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d29d59-af09-4187-a866-b357284242fb None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:33.875957 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 524/1063] 10.222.0.64 () {56 vars in 1109 bytes} [Tue Aug 2 09:47:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:47:33.881434 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d29d59-af09-4187-a866-b357284242fb None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:33.897757 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65145abe-7ff0-48b9-87af-5e5f877ff795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:33.898464 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65145abe-7ff0-48b9-87af-5e5f877ff795 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:33.898884 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65145abe-7ff0-48b9-87af-5e5f877ff795 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:33.899175 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65145abe-7ff0-48b9-87af-5e5f877ff795 None admin] PATH_INFO: `/v3/services` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:33.899950 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65145abe-7ff0-48b9-87af-5e5f877ff795 None admin] RBAC: Adding query filter params (type=image) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:47:33.900234 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65145abe-7ff0-48b9-87af-5e5f877ff795 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:33.901763 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65145abe-7ff0-48b9-87af-5e5f877ff795 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:33.906837 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 540/1064] 10.222.0.64 () {56 vars in 1108 bytes} [Tue Aug 2 09:47:33 2022] GET /identity/v3/services?type=image => generated 350 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:47:33.912388 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65145abe-7ff0-48b9-87af-5e5f877ff795 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:33.998252 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d29d59-af09-4187-a866-b357284242fb None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:34.004336 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:34.019466 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65145abe-7ff0-48b9-87af-5e5f877ff795 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:34.038723 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e578ebdd-91e9-4c38-b3d7-3ff95ccca538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:34.039686 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e578ebdd-91e9-4c38-b3d7-3ff95ccca538 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:34.040179 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e578ebdd-91e9-4c38-b3d7-3ff95ccca538 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:34.040668 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e578ebdd-91e9-4c38-b3d7-3ff95ccca538 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:34.048808 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e578ebdd-91e9-4c38-b3d7-3ff95ccca538 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:34.050526 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e578ebdd-91e9-4c38-b3d7-3ff95ccca538 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:34.053360 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8385868-8062-49f4-b479-cb1f9c431537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:34.053918 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8385868-8062-49f4-b479-cb1f9c431537 None admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:34.054201 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8385868-8062-49f4-b479-cb1f9c431537 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:34.054478 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8385868-8062-49f4-b479-cb1f9c431537 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:34.055181 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8385868-8062-49f4-b479-cb1f9c431537 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:34.056574 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8385868-8062-49f4-b479-cb1f9c431537 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:34.066261 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-e578ebdd-91e9-4c38-b3d7-3ff95ccca538 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 02 09:47:34.066261 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:47:34.066261 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:47:34.066261 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:47:34.066261 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:47:34.066261 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:47:34.066261 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:47:34.066261 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:47:34.066261 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:47:34.066261 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:47:34.066261 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:47:34.066261 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:47:34.066261 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 09:47:34.066261 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 09:47:34.066261 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 09:47:34.066261 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 09:47:34.068050 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:47:34.068050 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:47:34.068050 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 09:47:34.068050 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 09:47:34.068050 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 09:47:34.068050 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 09:47:34.068050 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:47:34.068050 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:47:34.068050 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:47:34.068050 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:47:34.068050 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:47:34.068050 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:47:34.068050 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:47:34.068050 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:47:34.068050 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:47:34.068050 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:47:34.068050 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:47:34.068050 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:47:34.069053 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:47:34.069053 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:47:34.069053 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 09:47:34.069053 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 09:47:34.069053 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 09:47:34.069053 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 09:47:34.069053 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 09:47:34.069053 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 09:47:34.069053 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 02 09:47:34.069053 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:47:34.069605 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 525/1065] 10.222.0.64 () {56 vars in 1093 bytes} [Tue Aug 2 09:47:33 2022] GET /identity/v3/users/nova => generated 82 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 09:47:34.075532 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e578ebdd-91e9-4c38-b3d7-3ff95ccca538 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:34.079563 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:34.092446 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 541/1066] 10.222.0.64 () {60 vars in 1144 bytes} [Tue Aug 2 09:47:33 2022] POST /identity/v3/endpoints => generated 327 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 09:47:34.194743 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e578ebdd-91e9-4c38-b3d7-3ff95ccca538 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:34.244233 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dc9ee5-84d1-4c2d-96ef-d748a87641e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:34.244876 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dc9ee5-84d1-4c2d-96ef-d748a87641e2 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:34.245191 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dc9ee5-84d1-4c2d-96ef-d748a87641e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:34.245427 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dc9ee5-84d1-4c2d-96ef-d748a87641e2 None admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:34.246350 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dc9ee5-84d1-4c2d-96ef-d748a87641e2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:47:34.246665 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dc9ee5-84d1-4c2d-96ef-d748a87641e2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:34.248211 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dc9ee5-84d1-4c2d-96ef-d748a87641e2 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:34.267990 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 526/1067] 10.222.0.64 () {56 vars in 1133 bytes} [Tue Aug 2 09:47:34 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:47:34.275031 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8385868-8062-49f4-b479-cb1f9c431537 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:34.385220 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8385868-8062-49f4-b479-cb1f9c431537 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:34.421313 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69e7ca8-0ef0-4f99-9158-4fc0b128b6eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:34.421959 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69e7ca8-0ef0-4f99-9158-4fc0b128b6eb None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:34.422245 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69e7ca8-0ef0-4f99-9158-4fc0b128b6eb None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:34.422523 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69e7ca8-0ef0-4f99-9158-4fc0b128b6eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:34.433521 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69e7ca8-0ef0-4f99-9158-4fc0b128b6eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:34.434984 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69e7ca8-0ef0-4f99-9158-4fc0b128b6eb None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:34.436891 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:34.452190 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-d69e7ca8-0ef0-4f99-9158-4fc0b128b6eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:47:34.452190 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:47:34.452190 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:47:34.452190 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:47:34.452190 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:47:34.452190 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:47:34.452190 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:47:34.452190 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:47:34.452190 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:47:34.452190 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:47:34.452190 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:47:34.452190 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:47:34.452190 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 09:47:34.452190 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 09:47:34.452190 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 09:47:34.453205 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 09:47:34.453205 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:47:34.453205 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:47:34.453205 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:47:34.453205 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:47:34.453205 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:47:34.453205 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:47:34.453205 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:47:34.453205 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:47:34.453205 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:47:34.453205 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:47:34.453205 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:47:34.453205 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:47:34.453205 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:47:34.453205 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:47:34.453205 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:47:34.453205 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:47:34.454127 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 09:47:34.454127 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 09:47:34.454127 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:47:34.454127 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:47:34.454929 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 542/1068] 10.222.0.64 () {56 vars in 1108 bytes} [Tue Aug 2 09:47:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:47:34.459393 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dc9ee5-84d1-4c2d-96ef-d748a87641e2 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:34.517326 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:34.562513 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dc9ee5-84d1-4c2d-96ef-d748a87641e2 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:34.600764 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33af7cb-7ed9-49ba-b933-7e6b730fcb25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:34.601450 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33af7cb-7ed9-49ba-b933-7e6b730fcb25 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:34.601738 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33af7cb-7ed9-49ba-b933-7e6b730fcb25 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:34.602012 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33af7cb-7ed9-49ba-b933-7e6b730fcb25 None admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:34.602713 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33af7cb-7ed9-49ba-b933-7e6b730fcb25 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:47:34.603023 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33af7cb-7ed9-49ba-b933-7e6b730fcb25 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:34.604422 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33af7cb-7ed9-49ba-b933-7e6b730fcb25 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:34.616925 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 527/1069] 10.222.0.64 () {56 vars in 1109 bytes} [Tue Aug 2 09:47:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:47:34.622454 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69e7ca8-0ef0-4f99-9158-4fc0b128b6eb None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:34.760695 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69e7ca8-0ef0-4f99-9158-4fc0b128b6eb None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:34.816987 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88026af-bf2a-4267-a4d1-61486c6b89b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:34.818086 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88026af-bf2a-4267-a4d1-61486c6b89b0 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:34.818590 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88026af-bf2a-4267-a4d1-61486c6b89b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:34.819096 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88026af-bf2a-4267-a4d1-61486c6b89b0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:34.831995 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88026af-bf2a-4267-a4d1-61486c6b89b0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:34.834809 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88026af-bf2a-4267-a4d1-61486c6b89b0 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:34.852202 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-c88026af-bf2a-4267-a4d1-61486c6b89b0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 09:47:34.852202 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:47:34.852202 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:47:34.852202 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:47:34.852202 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:47:34.852202 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:47:34.852202 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:47:34.852202 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:47:34.852202 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:47:34.852202 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:47:34.852202 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:47:34.852202 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:47:34.852202 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 09:47:34.852202 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 09:47:34.852202 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 09:47:34.853165 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 09:47:34.853165 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:47:34.853165 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:47:34.853165 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:47:34.853165 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:47:34.853165 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:47:34.853165 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:47:34.853165 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:47:34.853165 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:47:34.853165 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:47:34.853165 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:47:34.853165 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:47:34.853165 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:47:34.853165 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:47:34.853165 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:47:34.853165 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:47:34.853165 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:47:34.859131 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 09:47:34.859131 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 09:47:34.859131 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 09:47:34.859131 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 09:47:34.859131 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 09:47:34.859131 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 09:47:34.859131 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 09:47:34.859131 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:47:34.859131 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 543/1070] 10.222.0.64 () {56 vars in 1111 bytes} [Tue Aug 2 09:47:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:47:34.861010 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33af7cb-7ed9-49ba-b933-7e6b730fcb25 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:34.967710 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33af7cb-7ed9-49ba-b933-7e6b730fcb25 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:34.983876 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:35.002798 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fd9027-8765-4f43-9884-d91855c4d8f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:35.003420 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fd9027-8765-4f43-9884-d91855c4d8f7 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:35.003706 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fd9027-8765-4f43-9884-d91855c4d8f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:35.003985 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fd9027-8765-4f43-9884-d91855c4d8f7 None admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:35.004771 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fd9027-8765-4f43-9884-d91855c4d8f7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:47:35.005083 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fd9027-8765-4f43-9884-d91855c4d8f7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:35.006508 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fd9027-8765-4f43-9884-d91855c4d8f7 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:35.024178 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 528/1071] 10.222.0.64 () {56 vars in 1148 bytes} [Tue Aug 2 09:47:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:47:35.025058 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88026af-bf2a-4267-a4d1-61486c6b89b0 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:35.080380 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:35.126232 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88026af-bf2a-4267-a4d1-61486c6b89b0 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:35.160858 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1498b8-53b4-417e-a8eb-17d48fa5e4d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:35.161538 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1498b8-53b4-417e-a8eb-17d48fa5e4d7 None admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:35.161823 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1498b8-53b4-417e-a8eb-17d48fa5e4d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:35.162101 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1498b8-53b4-417e-a8eb-17d48fa5e4d7 None admin] PATH_INFO: `/v3/projects/390bd7c4f91c4465b0c7ab0f9e436c4d/users/99655e1f9ca24472a86cd26b0dbd1fc2/roles/55b4c595b54f40c6b03ed7161e5258bf` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:35.164307 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1498b8-53b4-417e-a8eb-17d48fa5e4d7 None admin] RBAC: Authorizing `identity:create_grant(project_id=390bd7c4f91c4465b0c7ab0f9e436c4d, user_id=99655e1f9ca24472a86cd26b0dbd1fc2, role_id=55b4c595b54f40c6b03ed7161e5258bf)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:35.165789 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1498b8-53b4-417e-a8eb-17d48fa5e4d7 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:35.180556 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 544/1072] 10.222.0.64 () {58 vars in 1439 bytes} [Tue Aug 2 09:47:35 2022] PUT /identity/v3/projects/390bd7c4f91c4465b0c7ab0f9e436c4d/users/99655e1f9ca24472a86cd26b0dbd1fc2/roles/55b4c595b54f40c6b03ed7161e5258bf => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:47:35.592707 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1987bc2e-19fd-4fb2-b431-99b9ec5e9552 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:35.593252 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1987bc2e-19fd-4fb2-b431-99b9ec5e9552 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:35.593252 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1987bc2e-19fd-4fb2-b431-99b9ec5e9552 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:35.594752 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 529/1073] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:47:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:47:35.600128 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892644a9-fbd6-485a-960f-56ee6ad00993 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:35.600562 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892644a9-fbd6-485a-960f-56ee6ad00993 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:35.600859 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892644a9-fbd6-485a-960f-56ee6ad00993 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:35.619322 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-892644a9-fbd6-485a-960f-56ee6ad00993 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:47:35.620532 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-892644a9-fbd6-485a-960f-56ee6ad00993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:47:35.699308 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 545/1074] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:47:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 09:47:35.721664 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1987bc2e-19fd-4fb2-b431-99b9ec5e9552 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:35.839514 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1987bc2e-19fd-4fb2-b431-99b9ec5e9552 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:35.887152 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4798c2-df6b-47b3-af58-4a514b95292d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:35.888168 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4798c2-df6b-47b3-af58-4a514b95292d None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:35.888776 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4798c2-df6b-47b3-af58-4a514b95292d None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:35.888953 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4798c2-df6b-47b3-af58-4a514b95292d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:35.899943 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4798c2-df6b-47b3-af58-4a514b95292d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 09:47:35.905473 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-9e4798c2-df6b-47b3-af58-4a514b95292d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 09:47:35.905473 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:47:35.905473 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:47:35.905473 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:47:35.905473 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:47:35.905473 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:47:35.905473 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:47:35.905473 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:47:35.905473 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:47:35.905473 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:47:35.905473 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:47:35.905473 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:47:35.905473 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 09:47:35.905473 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 09:47:35.905473 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 09:47:35.905473 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 09:47:35.906416 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 09:47:35.906416 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 09:47:35.906416 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 09:47:35.906416 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 09:47:35.906416 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:47:35.906416 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:47:35.906416 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:47:35.906416 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:47:35.906416 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:47:35.906416 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:47:35.906416 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:47:35.906416 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:47:35.906416 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:47:35.906416 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:47:35.906416 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:47:35.906416 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:47:35.907312 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:47:35.907312 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:47:35.907312 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:47:35.907312 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:47:35.907312 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 09:47:35.907312 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 09:47:35.907312 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 09:47:35.907312 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 09:47:35.907312 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 09:47:35.907312 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 09:47:35.907312 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 09:47:35.907312 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:47:35.907312 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 530/1075] 10.222.0.64 () {56 vars in 1096 bytes} [Tue Aug 2 09:47:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:47:35.913993 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892644a9-fbd6-485a-960f-56ee6ad00993 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:35.978416 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:36.045448 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892644a9-fbd6-485a-960f-56ee6ad00993 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:36.083862 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681bca32-1139-4199-b70a-e53045d98f4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:36.084508 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681bca32-1139-4199-b70a-e53045d98f4a None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:36.084796 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681bca32-1139-4199-b70a-e53045d98f4a None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:36.085070 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681bca32-1139-4199-b70a-e53045d98f4a None admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:36.085786 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681bca32-1139-4199-b70a-e53045d98f4a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:47:36.086094 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681bca32-1139-4199-b70a-e53045d98f4a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:36.087467 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681bca32-1139-4199-b70a-e53045d98f4a None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:36.102433 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 546/1076] 10.222.0.64 () {56 vars in 1099 bytes} [Tue Aug 2 09:47:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:47:36.103876 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4798c2-df6b-47b3-af58-4a514b95292d None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:36.145747 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:36.158773 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90a84b6-1942-42ed-bcab-dd1bd1e6cebc None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:36.159347 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90a84b6-1942-42ed-bcab-dd1bd1e6cebc None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:36.159694 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90a84b6-1942-42ed-bcab-dd1bd1e6cebc None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:36.161419 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 547/1077] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:47:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:47:36.166288 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b32206a-fe8a-4881-9291-3abab384f833 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:36.166578 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b32206a-fe8a-4881-9291-3abab384f833 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:36.166910 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b32206a-fe8a-4881-9291-3abab384f833 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:36.186196 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-3b32206a-fe8a-4881-9291-3abab384f833 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:47:36.187571 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-3b32206a-fe8a-4881-9291-3abab384f833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:47:36.233739 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4798c2-df6b-47b3-af58-4a514b95292d None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:36.268619 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 548/1078] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:47:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 02 09:47:36.269587 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e027e489-299c-422b-8fff-d3725dafc52a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:36.270289 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e027e489-299c-422b-8fff-d3725dafc52a None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:36.270638 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e027e489-299c-422b-8fff-d3725dafc52a None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:36.270998 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e027e489-299c-422b-8fff-d3725dafc52a None admin] PATH_INFO: `/v3/users/9e98f79dd6114048a6a4683e6e0c8809` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:36.272223 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e027e489-299c-422b-8fff-d3725dafc52a None admin] RBAC: Authorizing `identity:get_user(user_id=9e98f79dd6114048a6a4683e6e0c8809)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:36.273760 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e027e489-299c-422b-8fff-d3725dafc52a None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:36.275576 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 531/1079] 10.222.0.64 () {56 vars in 1177 bytes} [Tue Aug 2 09:47:36 2022] GET /identity/v3/users/9e98f79dd6114048a6a4683e6e0c8809 => generated 285 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:47:36.280216 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b32206a-fe8a-4881-9291-3abab384f833 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:36.281845 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e027e489-299c-422b-8fff-d3725dafc52a None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:36.317568 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:36.395899 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b32206a-fe8a-4881-9291-3abab384f833 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:36.397272 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e027e489-299c-422b-8fff-d3725dafc52a None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:36.399579 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:36.437198 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b6262a-745e-46f3-add5-0802a9148ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:36.437824 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b6262a-745e-46f3-add5-0802a9148ab8 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:36.438107 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b6262a-745e-46f3-add5-0802a9148ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:36.438378 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b6262a-745e-46f3-add5-0802a9148ab8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:36.440995 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518997d6-0afc-487b-9ce9-cff512092b68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:36.441633 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518997d6-0afc-487b-9ce9-cff512092b68 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:36.441925 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518997d6-0afc-487b-9ce9-cff512092b68 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:36.442201 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518997d6-0afc-487b-9ce9-cff512092b68 None admin] PATH_INFO: `/v3/projects/e502fb4fefa64be9b2e40bdec3c99876` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:36.443361 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518997d6-0afc-487b-9ce9-cff512092b68 None admin] RBAC: Authorizing `identity:get_project(project_id=e502fb4fefa64be9b2e40bdec3c99876)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:36.444857 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518997d6-0afc-487b-9ce9-cff512092b68 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:36.446529 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 532/1080] 10.222.0.64 () {56 vars in 1186 bytes} [Tue Aug 2 09:47:36 2022] GET /identity/v3/projects/e502fb4fefa64be9b2e40bdec3c99876 => generated 311 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:47:36.452999 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518997d6-0afc-487b-9ce9-cff512092b68 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:36.510153 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b6262a-745e-46f3-add5-0802a9148ab8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:36.511600 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b6262a-745e-46f3-add5-0802a9148ab8 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:36.549274 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:36.568467 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518997d6-0afc-487b-9ce9-cff512092b68 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:36.615542 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff88294b-e439-4a75-88b3-c7d042862df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:36.616239 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff88294b-e439-4a75-88b3-c7d042862df2 None admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:36.616516 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff88294b-e439-4a75-88b3-c7d042862df2 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:36.616772 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff88294b-e439-4a75-88b3-c7d042862df2 None admin] PATH_INFO: `/v3/projects/e502fb4fefa64be9b2e40bdec3c99876/users/9e98f79dd6114048a6a4683e6e0c8809/roles/1000cf318fa14915b8091599eaead5de` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:36.618773 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff88294b-e439-4a75-88b3-c7d042862df2 None admin] RBAC: Authorizing `identity:create_grant(project_id=e502fb4fefa64be9b2e40bdec3c99876, user_id=9e98f79dd6114048a6a4683e6e0c8809, role_id=1000cf318fa14915b8091599eaead5de)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:36.620209 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff88294b-e439-4a75-88b3-c7d042862df2 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:36.633138 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 549/1081] 10.222.0.64 () {58 vars in 1282 bytes} [Tue Aug 2 09:47:36 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 02 09:47:36.636046 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 533/1082] 10.222.0.64 () {58 vars in 1439 bytes} [Tue Aug 2 09:47:36 2022] PUT /identity/v3/projects/e502fb4fefa64be9b2e40bdec3c99876/users/9e98f79dd6114048a6a4683e6e0c8809/roles/1000cf318fa14915b8091599eaead5de => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:47:36.641409 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b6262a-745e-46f3-add5-0802a9148ab8 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:36.764618 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b6262a-745e-46f3-add5-0802a9148ab8 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:36.804893 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fe07b7-7601-403c-8091-3c17eee65272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:36.805898 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fe07b7-7601-403c-8091-3c17eee65272 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:36.806249 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fe07b7-7601-403c-8091-3c17eee65272 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:36.806590 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fe07b7-7601-403c-8091-3c17eee65272 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:36.810252 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:36.819733 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fe07b7-7601-403c-8091-3c17eee65272 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:36.821344 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fe07b7-7601-403c-8091-3c17eee65272 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:36.839200 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-f4fe07b7-7601-403c-8091-3c17eee65272 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:47:36.839200 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:47:36.839200 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:47:36.839200 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:47:36.839200 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:47:36.839200 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:47:36.839200 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:47:36.839200 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:47:36.839200 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:47:36.839200 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:47:36.839200 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:47:36.839200 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:47:36.839200 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 09:47:36.839200 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 09:47:36.839200 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 09:47:36.840544 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 09:47:36.840544 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:47:36.840544 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:47:36.840544 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:47:36.840544 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:47:36.840544 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:47:36.840544 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:47:36.840544 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:47:36.840544 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:47:36.840544 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:47:36.840544 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:47:36.840544 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:47:36.840544 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:47:36.840544 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:47:36.840544 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:47:36.840544 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:47:36.840544 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:47:36.842694 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 09:47:36.842694 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 09:47:36.842694 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:47:36.842694 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:47:36.842694 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 550/1083] 10.222.0.64 () {56 vars in 1108 bytes} [Tue Aug 2 09:47:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:47:36.851928 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff88294b-e439-4a75-88b3-c7d042862df2 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:36.888577 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:36.962238 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff88294b-e439-4a75-88b3-c7d042862df2 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:37.001055 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c0291d-ea00-4d69-8cb7-f9a8e4bd6ca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:37.001860 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c0291d-ea00-4d69-8cb7-f9a8e4bd6ca1 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:37.002242 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c0291d-ea00-4d69-8cb7-f9a8e4bd6ca1 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:37.002613 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c0291d-ea00-4d69-8cb7-f9a8e4bd6ca1 None admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:37.003409 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c0291d-ea00-4d69-8cb7-f9a8e4bd6ca1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:47:37.003811 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c0291d-ea00-4d69-8cb7-f9a8e4bd6ca1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:37.005318 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c0291d-ea00-4d69-8cb7-f9a8e4bd6ca1 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:37.019121 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 534/1084] 10.222.0.64 () {56 vars in 1109 bytes} [Tue Aug 2 09:47:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:47:37.269655 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c78afe-8fd1-4804-a565-c045d917c5aa None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:37.270005 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c78afe-8fd1-4804-a565-c045d917c5aa None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:37.270207 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c78afe-8fd1-4804-a565-c045d917c5aa None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:37.271547 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 551/1085] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:47:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:47:37.277370 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6218b9ff-8989-4f1e-91be-35765cf76ea8 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:37.277666 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6218b9ff-8989-4f1e-91be-35765cf76ea8 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:37.277948 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6218b9ff-8989-4f1e-91be-35765cf76ea8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:37.298468 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-6218b9ff-8989-4f1e-91be-35765cf76ea8 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:47:37.300040 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-6218b9ff-8989-4f1e-91be-35765cf76ea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:47:37.382867 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 535/1086] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:47:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 09:47:37.405976 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c78afe-8fd1-4804-a565-c045d917c5aa None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:37.508460 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c78afe-8fd1-4804-a565-c045d917c5aa None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:37.541049 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b603c2e4-5438-4277-9b4a-ef4d12eb2665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:37.541610 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b603c2e4-5438-4277-9b4a-ef4d12eb2665 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:37.541890 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b603c2e4-5438-4277-9b4a-ef4d12eb2665 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:37.542163 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b603c2e4-5438-4277-9b4a-ef4d12eb2665 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:37.553664 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b603c2e4-5438-4277-9b4a-ef4d12eb2665 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 09:47:37.563433 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-b603c2e4-5438-4277-9b4a-ef4d12eb2665 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 02 09:47:37.563433 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:47:37.563433 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:47:37.563433 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:47:37.563433 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:47:37.563433 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:47:37.563433 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:47:37.563433 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:47:37.563433 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:47:37.563433 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:47:37.563433 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:47:37.563433 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:47:37.563433 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 09:47:37.563433 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 09:47:37.563433 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 09:47:37.563433 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 09:47:37.564483 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 09:47:37.564483 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 09:47:37.564483 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 09:47:37.564483 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 09:47:37.564483 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:47:37.564483 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:47:37.564483 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:47:37.564483 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:47:37.564483 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:47:37.564483 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:47:37.564483 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:47:37.564483 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:47:37.564483 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:47:37.564483 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:47:37.564483 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:47:37.564483 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:47:37.565475 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:47:37.565475 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:47:37.565475 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:47:37.565475 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:47:37.565475 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 09:47:37.565475 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 09:47:37.565475 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 09:47:37.565475 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 09:47:37.565475 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 09:47:37.565475 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 09:47:37.565475 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 02 09:47:37.565475 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:47:37.566218 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 552/1087] 10.222.0.64 () {56 vars in 1120 bytes} [Tue Aug 2 09:47:37 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:47:37.571517 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6218b9ff-8989-4f1e-91be-35765cf76ea8 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:37.594860 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:37.670435 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6218b9ff-8989-4f1e-91be-35765cf76ea8 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:37.703836 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a791fa0c-7593-4963-84ba-59dbf542b6b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:37.704480 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a791fa0c-7593-4963-84ba-59dbf542b6b3 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:37.704772 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a791fa0c-7593-4963-84ba-59dbf542b6b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:37.705050 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a791fa0c-7593-4963-84ba-59dbf542b6b3 None admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:37.705796 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a791fa0c-7593-4963-84ba-59dbf542b6b3 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:47:37.706100 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a791fa0c-7593-4963-84ba-59dbf542b6b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:37.707460 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a791fa0c-7593-4963-84ba-59dbf542b6b3 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:37.715265 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 536/1088] 10.222.0.64 () {56 vars in 1115 bytes} [Tue Aug 2 09:47:37 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:47:37.720894 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b603c2e4-5438-4277-9b4a-ef4d12eb2665 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:37.770299 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:37.822645 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b603c2e4-5438-4277-9b4a-ef4d12eb2665 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:37.862269 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d79e847-1140-4345-9c71-4200fccad6a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:37.862997 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d79e847-1140-4345-9c71-4200fccad6a2 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:37.863330 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d79e847-1140-4345-9c71-4200fccad6a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:37.863657 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d79e847-1140-4345-9c71-4200fccad6a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:37.875304 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d79e847-1140-4345-9c71-4200fccad6a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:37.876818 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d79e847-1140-4345-9c71-4200fccad6a2 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:37.897065 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-2d79e847-1140-4345-9c71-4200fccad6a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:47:37.897065 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:47:37.897065 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:47:37.897065 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:47:37.897065 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:47:37.897065 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:47:37.897065 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:47:37.897065 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:47:37.897065 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:47:37.897065 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:47:37.897065 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:47:37.897065 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:47:37.897065 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 09:47:37.897065 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 09:47:37.897065 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 09:47:37.898208 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 09:47:37.898208 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:47:37.898208 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:47:37.898208 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:47:37.898208 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:47:37.898208 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:47:37.898208 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:47:37.898208 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:47:37.898208 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:47:37.898208 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:47:37.898208 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:47:37.898208 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:47:37.898208 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:47:37.898208 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:47:37.898208 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:47:37.898208 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:47:37.898208 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:47:37.899167 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 09:47:37.899167 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 09:47:37.899167 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:47:37.899167 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:47:37.899167 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 553/1089] 10.222.0.64 () {56 vars in 1108 bytes} [Tue Aug 2 09:47:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 09:47:37.904905 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a791fa0c-7593-4963-84ba-59dbf542b6b3 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:37.991919 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:38.009235 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a791fa0c-7593-4963-84ba-59dbf542b6b3 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:38.042931 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781383bf-475d-41c4-b379-a9820d91fa83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:38.043546 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781383bf-475d-41c4-b379-a9820d91fa83 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:38.043830 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781383bf-475d-41c4-b379-a9820d91fa83 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:38.044102 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781383bf-475d-41c4-b379-a9820d91fa83 None admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:38.044830 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781383bf-475d-41c4-b379-a9820d91fa83 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:47:38.045681 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781383bf-475d-41c4-b379-a9820d91fa83 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:38.047052 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781383bf-475d-41c4-b379-a9820d91fa83 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:38.059113 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 537/1090] 10.222.0.64 () {56 vars in 1109 bytes} [Tue Aug 2 09:47:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:47:38.065269 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d79e847-1140-4345-9c71-4200fccad6a2 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:38.161342 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:38.164636 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d79e847-1140-4345-9c71-4200fccad6a2 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:38.199044 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d3f54d-974a-4674-bd11-719fd506e481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:38.199680 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d3f54d-974a-4674-bd11-719fd506e481 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:38.199962 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d3f54d-974a-4674-bd11-719fd506e481 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:38.200254 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d3f54d-974a-4674-bd11-719fd506e481 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:38.208339 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d3f54d-974a-4674-bd11-719fd506e481 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:38.209773 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d3f54d-974a-4674-bd11-719fd506e481 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:38.220782 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-b8d3f54d-974a-4674-bd11-719fd506e481 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 02 09:47:38.220782 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:47:38.220782 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:47:38.220782 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:47:38.220782 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:47:38.220782 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:47:38.220782 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:47:38.220782 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:47:38.220782 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:47:38.220782 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:47:38.220782 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:47:38.220782 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:47:38.220782 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 09:47:38.220782 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 09:47:38.220782 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 09:47:38.220782 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 09:47:38.221942 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:47:38.221942 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:47:38.221942 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 09:47:38.221942 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 09:47:38.221942 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 09:47:38.221942 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 09:47:38.221942 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:47:38.221942 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:47:38.221942 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:47:38.221942 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:47:38.221942 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:47:38.221942 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:47:38.221942 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:47:38.221942 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:47:38.221942 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:47:38.221942 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:47:38.221942 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:47:38.221942 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:47:38.222949 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:47:38.222949 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:47:38.222949 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 09:47:38.222949 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 09:47:38.222949 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 09:47:38.222949 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 09:47:38.222949 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 09:47:38.222949 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 09:47:38.222949 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 02 09:47:38.222949 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:47:38.222949 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 554/1091] 10.222.0.64 () {56 vars in 1093 bytes} [Tue Aug 2 09:47:38 2022] GET /identity/v3/users/nova => generated 82 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:47:38.227944 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781383bf-475d-41c4-b379-a9820d91fa83 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:38.327807 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781383bf-475d-41c4-b379-a9820d91fa83 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:38.362452 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5ac12c-c595-4d2a-bd8f-6a00a7101377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:38.363077 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5ac12c-c595-4d2a-bd8f-6a00a7101377 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:38.363360 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5ac12c-c595-4d2a-bd8f-6a00a7101377 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:38.363633 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5ac12c-c595-4d2a-bd8f-6a00a7101377 None admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:38.364461 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5ac12c-c595-4d2a-bd8f-6a00a7101377 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:47:38.364767 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5ac12c-c595-4d2a-bd8f-6a00a7101377 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:38.366191 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5ac12c-c595-4d2a-bd8f-6a00a7101377 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:38.384688 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 538/1092] 10.222.0.64 () {56 vars in 1133 bytes} [Tue Aug 2 09:47:38 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:47:38.390696 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d3f54d-974a-4674-bd11-719fd506e481 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:38.495782 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d3f54d-974a-4674-bd11-719fd506e481 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:38.534809 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccd152a-ef25-4966-9e2c-8d8e3475dc98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:38.535432 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccd152a-ef25-4966-9e2c-8d8e3475dc98 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:38.535718 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccd152a-ef25-4966-9e2c-8d8e3475dc98 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:38.535995 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccd152a-ef25-4966-9e2c-8d8e3475dc98 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:38.566266 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:38.609582 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccd152a-ef25-4966-9e2c-8d8e3475dc98 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:38.611048 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccd152a-ef25-4966-9e2c-8d8e3475dc98 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:38.676291 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6c8219-858a-4ec2-b915-4d2671a4be8b None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:38.676819 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6c8219-858a-4ec2-b915-4d2671a4be8b None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:38.676819 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6c8219-858a-4ec2-b915-4d2671a4be8b None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:38.678206 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 539/1093] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:47:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 09:47:38.683722 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c63263e-6129-45db-bae5-69fd5c792ae9 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:38.684345 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c63263e-6129-45db-bae5-69fd5c792ae9 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:38.684733 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c63263e-6129-45db-bae5-69fd5c792ae9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:38.691135 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:38.703786 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-7c63263e-6129-45db-bae5-69fd5c792ae9 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:47:38.705256 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-7c63263e-6129-45db-bae5-69fd5c792ae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:47:38.710987 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 555/1094] 10.222.0.64 () {58 vars in 1282 bytes} [Tue Aug 2 09:47:38 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 02 09:47:38.717258 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccd152a-ef25-4966-9e2c-8d8e3475dc98 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:38.775243 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:38.775889 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 540/1095] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:47:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 02 09:47:38.799523 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c63263e-6129-45db-bae5-69fd5c792ae9 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:38.816273 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccd152a-ef25-4966-9e2c-8d8e3475dc98 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:38.852849 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbf2412-f994-4715-83b2-0ed8f5ccb8c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:38.853414 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbf2412-f994-4715-83b2-0ed8f5ccb8c4 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:38.853695 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbf2412-f994-4715-83b2-0ed8f5ccb8c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:38.853966 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbf2412-f994-4715-83b2-0ed8f5ccb8c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:38.865073 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbf2412-f994-4715-83b2-0ed8f5ccb8c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:38.866525 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbf2412-f994-4715-83b2-0ed8f5ccb8c4 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:38.881147 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-adbf2412-f994-4715-83b2-0ed8f5ccb8c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:47:38.881147 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:47:38.881147 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:47:38.881147 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:47:38.881147 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:47:38.881147 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:47:38.881147 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:47:38.881147 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:47:38.881147 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:47:38.881147 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:47:38.881147 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:47:38.881147 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:47:38.881147 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 09:47:38.881147 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 09:47:38.881147 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 09:47:38.882232 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 09:47:38.882232 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:47:38.882232 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:47:38.882232 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:47:38.882232 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:47:38.882232 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:47:38.882232 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:47:38.882232 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:47:38.882232 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:47:38.882232 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:47:38.882232 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:47:38.882232 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:47:38.882232 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:47:38.882232 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:47:38.882232 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:47:38.882232 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:47:38.882232 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:47:38.883198 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 09:47:38.883198 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 09:47:38.883198 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:47:38.883198 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:47:38.883198 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 556/1096] 10.222.0.64 () {56 vars in 1108 bytes} [Tue Aug 2 09:47:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 09:47:38.887931 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbf2412-f994-4715-83b2-0ed8f5ccb8c4 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:38.906290 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c63263e-6129-45db-bae5-69fd5c792ae9 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:38.936542 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:38.941948 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdd83c9-bd0b-4116-997c-2bb6d604eadd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:38.942596 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdd83c9-bd0b-4116-997c-2bb6d604eadd None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:38.942879 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdd83c9-bd0b-4116-997c-2bb6d604eadd None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:38.943153 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdd83c9-bd0b-4116-997c-2bb6d604eadd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:38.954043 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdd83c9-bd0b-4116-997c-2bb6d604eadd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 09:47:38.958905 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-8cdd83c9-bd0b-4116-997c-2bb6d604eadd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 09:47:38.958905 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:47:38.958905 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:47:38.958905 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:47:38.958905 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:47:38.958905 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:47:38.958905 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:47:38.958905 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:47:38.958905 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:47:38.958905 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:47:38.958905 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:47:38.958905 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:47:38.958905 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 09:47:38.958905 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 09:47:38.958905 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 09:47:38.958905 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 09:47:38.965157 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 09:47:38.965157 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 09:47:38.965157 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 09:47:38.965157 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 09:47:38.965157 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:47:38.965157 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:47:38.965157 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:47:38.965157 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:47:38.965157 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:47:38.965157 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:47:38.965157 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:47:38.965157 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:47:38.965157 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:47:38.965157 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:47:38.965157 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:47:38.965157 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:47:38.966142 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:47:38.966142 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:47:38.966142 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:47:38.966142 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:47:38.966142 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 09:47:38.966142 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 09:47:38.966142 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 09:47:38.966142 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 09:47:38.966142 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 09:47:38.966142 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 09:47:38.966142 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 09:47:38.966142 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:47:38.966142 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 541/1097] 10.222.0.64 () {56 vars in 1096 bytes} [Tue Aug 2 09:47:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 09:47:38.967467 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdd83c9-bd0b-4116-997c-2bb6d604eadd None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:38.989114 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbf2412-f994-4715-83b2-0ed8f5ccb8c4 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:39.003003 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:39.026383 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fc1a28-6d9c-43e9-8f6b-5871be4a7e8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:39.026934 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fc1a28-6d9c-43e9-8f6b-5871be4a7e8d None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:39.027218 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fc1a28-6d9c-43e9-8f6b-5871be4a7e8d None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:39.027492 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fc1a28-6d9c-43e9-8f6b-5871be4a7e8d None admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:39.028187 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fc1a28-6d9c-43e9-8f6b-5871be4a7e8d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:47:39.028544 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fc1a28-6d9c-43e9-8f6b-5871be4a7e8d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:39.030462 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fc1a28-6d9c-43e9-8f6b-5871be4a7e8d None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:39.042293 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 557/1098] 10.222.0.64 () {56 vars in 1109 bytes} [Tue Aug 2 09:47:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:47:39.047950 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fc1a28-6d9c-43e9-8f6b-5871be4a7e8d None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:39.071475 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdd83c9-bd0b-4116-997c-2bb6d604eadd None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:39.106685 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccfeb5a-d2c1-4cbf-8d54-7ab596ca66b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:39.107307 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccfeb5a-d2c1-4cbf-8d54-7ab596ca66b5 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:39.107650 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccfeb5a-d2c1-4cbf-8d54-7ab596ca66b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:39.107926 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccfeb5a-d2c1-4cbf-8d54-7ab596ca66b5 None admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:39.108675 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccfeb5a-d2c1-4cbf-8d54-7ab596ca66b5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:47:39.108981 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccfeb5a-d2c1-4cbf-8d54-7ab596ca66b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:39.110343 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccfeb5a-d2c1-4cbf-8d54-7ab596ca66b5 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:39.118679 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 542/1099] 10.222.0.64 () {56 vars in 1099 bytes} [Tue Aug 2 09:47:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:47:39.124457 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccfeb5a-d2c1-4cbf-8d54-7ab596ca66b5 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:39.146127 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fc1a28-6d9c-43e9-8f6b-5871be4a7e8d None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:39.179529 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:39.179892 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6cfcf1-c0e7-4b6a-8b78-a81fffb7bc71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:39.180565 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6cfcf1-c0e7-4b6a-8b78-a81fffb7bc71 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:39.180849 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6cfcf1-c0e7-4b6a-8b78-a81fffb7bc71 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:39.181130 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6cfcf1-c0e7-4b6a-8b78-a81fffb7bc71 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:39.188700 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6cfcf1-c0e7-4b6a-8b78-a81fffb7bc71 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:39.190126 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6cfcf1-c0e7-4b6a-8b78-a81fffb7bc71 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:39.201227 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-8e6cfcf1-c0e7-4b6a-8b78-a81fffb7bc71 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 09:47:39.201227 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:47:39.201227 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:47:39.201227 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:47:39.201227 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:47:39.201227 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:47:39.201227 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:47:39.201227 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:47:39.201227 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:47:39.201227 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:47:39.201227 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:47:39.201227 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:47:39.201227 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 09:47:39.201227 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 09:47:39.201227 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 09:47:39.202348 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 09:47:39.202348 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:47:39.202348 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:47:39.202348 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:47:39.202348 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:47:39.202348 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:47:39.202348 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:47:39.202348 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:47:39.202348 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:47:39.202348 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:47:39.202348 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:47:39.202348 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:47:39.202348 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:47:39.202348 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:47:39.202348 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:47:39.202348 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:47:39.202348 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:47:39.203708 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 09:47:39.203708 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 09:47:39.203708 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 09:47:39.203708 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 09:47:39.203708 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 09:47:39.203708 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 09:47:39.203708 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 09:47:39.203708 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:47:39.203708 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 558/1100] 10.222.0.64 () {56 vars in 1111 bytes} [Tue Aug 2 09:47:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 09:47:39.207987 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6cfcf1-c0e7-4b6a-8b78-a81fffb7bc71 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:39.226827 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccfeb5a-d2c1-4cbf-8d54-7ab596ca66b5 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:39.238072 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:39.261448 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67d3d87-1fbf-4f01-afd7-3d31e1671219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:39.262086 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67d3d87-1fbf-4f01-afd7-3d31e1671219 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:39.262366 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67d3d87-1fbf-4f01-afd7-3d31e1671219 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:39.262641 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67d3d87-1fbf-4f01-afd7-3d31e1671219 None admin] PATH_INFO: `/v3/users/9e98f79dd6114048a6a4683e6e0c8809` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:39.263835 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67d3d87-1fbf-4f01-afd7-3d31e1671219 None admin] RBAC: Authorizing `identity:get_user(user_id=9e98f79dd6114048a6a4683e6e0c8809)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:39.265375 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67d3d87-1fbf-4f01-afd7-3d31e1671219 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:39.266973 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 543/1101] 10.222.0.64 () {56 vars in 1177 bytes} [Tue Aug 2 09:47:39 2022] GET /identity/v3/users/9e98f79dd6114048a6a4683e6e0c8809 => generated 285 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:47:39.272910 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67d3d87-1fbf-4f01-afd7-3d31e1671219 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:39.306939 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6cfcf1-c0e7-4b6a-8b78-a81fffb7bc71 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:39.339830 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3cbef3-f564-441b-ba41-d678d03ee0aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:39.340489 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3cbef3-f564-441b-ba41-d678d03ee0aa None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:39.340774 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3cbef3-f564-441b-ba41-d678d03ee0aa None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:39.341047 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3cbef3-f564-441b-ba41-d678d03ee0aa None admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:39.342454 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3cbef3-f564-441b-ba41-d678d03ee0aa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:47:39.342920 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3cbef3-f564-441b-ba41-d678d03ee0aa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:39.344462 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3cbef3-f564-441b-ba41-d678d03ee0aa None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:39.359513 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 559/1102] 10.222.0.64 () {56 vars in 1148 bytes} [Tue Aug 2 09:47:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:47:39.365725 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3cbef3-f564-441b-ba41-d678d03ee0aa None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:39.384526 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67d3d87-1fbf-4f01-afd7-3d31e1671219 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:39.421992 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:39.440091 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7be2a55-71ef-4fb2-9857-2780d59bdab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:39.441175 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7be2a55-71ef-4fb2-9857-2780d59bdab7 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:39.441604 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7be2a55-71ef-4fb2-9857-2780d59bdab7 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:39.442082 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7be2a55-71ef-4fb2-9857-2780d59bdab7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:39.463974 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:39.475963 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3cbef3-f564-441b-ba41-d678d03ee0aa None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:39.511276 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4886b5-2d3a-4a24-a5bc-7fe5a2fa3458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:39.511891 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4886b5-2d3a-4a24-a5bc-7fe5a2fa3458 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:39.512169 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4886b5-2d3a-4a24-a5bc-7fe5a2fa3458 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:39.512480 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4886b5-2d3a-4a24-a5bc-7fe5a2fa3458 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:39.513245 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4886b5-2d3a-4a24-a5bc-7fe5a2fa3458 None admin] RBAC: Adding query filter params (role.id=55b4c595b54f40c6b03ed7161e5258bf, scope.project.id=390bd7c4f91c4465b0c7ab0f9e436c4d, user.id=99655e1f9ca24472a86cd26b0dbd1fc2) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:47:39.513573 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4886b5-2d3a-4a24-a5bc-7fe5a2fa3458 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:39.515132 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4886b5-2d3a-4a24-a5bc-7fe5a2fa3458 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:39.520908 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7be2a55-71ef-4fb2-9857-2780d59bdab7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:39.521077 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 560/1103] 10.222.0.64 () {56 vars in 1374 bytes} [Tue Aug 2 09:47:39 2022] GET /identity/v3/role_assignments?user.id=99655e1f9ca24472a86cd26b0dbd1fc2&scope.project.id=390bd7c4f91c4465b0c7ab0f9e436c4d&role.id=55b4c595b54f40c6b03ed7161e5258bf => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:47:39.522465 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7be2a55-71ef-4fb2-9857-2780d59bdab7 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:39.646706 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 544/1104] 10.222.0.64 () {58 vars in 1282 bytes} [Tue Aug 2 09:47:39 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 02 09:47:39.652893 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4886b5-2d3a-4a24-a5bc-7fe5a2fa3458 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:39.782076 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4886b5-2d3a-4a24-a5bc-7fe5a2fa3458 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:39.817140 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7355c3a-d042-4068-a071-e2a67c067a17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:39.817782 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7355c3a-d042-4068-a071-e2a67c067a17 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:39.818068 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7355c3a-d042-4068-a071-e2a67c067a17 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:39.818367 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7355c3a-d042-4068-a071-e2a67c067a17 None admin] PATH_INFO: `/v3/projects/e502fb4fefa64be9b2e40bdec3c99876` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:39.819498 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7355c3a-d042-4068-a071-e2a67c067a17 None admin] RBAC: Authorizing `identity:get_project(project_id=e502fb4fefa64be9b2e40bdec3c99876)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:39.820964 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7355c3a-d042-4068-a071-e2a67c067a17 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:39.822630 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 561/1105] 10.222.0.64 () {56 vars in 1186 bytes} [Tue Aug 2 09:47:39 2022] GET /identity/v3/projects/e502fb4fefa64be9b2e40bdec3c99876 => generated 311 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:47:39.829009 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7be2a55-71ef-4fb2-9857-2780d59bdab7 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:39.854443 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:39.932792 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7be2a55-71ef-4fb2-9857-2780d59bdab7 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:39.972647 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f63efe4-115b-4f67-b8c9-10ae7a700670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:39.973530 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f63efe4-115b-4f67-b8c9-10ae7a700670 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:39.973833 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f63efe4-115b-4f67-b8c9-10ae7a700670 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:39.974165 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f63efe4-115b-4f67-b8c9-10ae7a700670 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:39.974975 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f63efe4-115b-4f67-b8c9-10ae7a700670 None admin] RBAC: Adding query filter params (role.id=1000cf318fa14915b8091599eaead5de, scope.project.id=e502fb4fefa64be9b2e40bdec3c99876, user.id=9e98f79dd6114048a6a4683e6e0c8809) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:47:39.975290 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f63efe4-115b-4f67-b8c9-10ae7a700670 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:39.976797 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f63efe4-115b-4f67-b8c9-10ae7a700670 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:39.983402 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 545/1106] 10.222.0.64 () {56 vars in 1374 bytes} [Tue Aug 2 09:47:39 2022] GET /identity/v3/role_assignments?user.id=9e98f79dd6114048a6a4683e6e0c8809&scope.project.id=e502fb4fefa64be9b2e40bdec3c99876&role.id=1000cf318fa14915b8091599eaead5de => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:47:42.018443 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827fd313-5dfb-4595-a3c8-141b36f7f137 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:42.018981 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827fd313-5dfb-4595-a3c8-141b36f7f137 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:42.018981 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827fd313-5dfb-4595-a3c8-141b36f7f137 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:42.020371 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 562/1107] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:47:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:47:42.025528 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2599862-5b87-4276-846e-c7716f2ed84b None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:42.025898 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2599862-5b87-4276-846e-c7716f2ed84b None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:42.026088 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2599862-5b87-4276-846e-c7716f2ed84b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:42.044353 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-d2599862-5b87-4276-846e-c7716f2ed84b None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:47:42.045617 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-d2599862-5b87-4276-846e-c7716f2ed84b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:47:42.110754 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 546/1108] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:47:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 09:47:42.134759 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827fd313-5dfb-4595-a3c8-141b36f7f137 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:42.159562 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:42.234667 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827fd313-5dfb-4595-a3c8-141b36f7f137 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:42.267843 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79518ef-5761-4f9f-bfd0-60180ced7da0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:42.268505 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79518ef-5761-4f9f-bfd0-60180ced7da0 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:42.268788 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79518ef-5761-4f9f-bfd0-60180ced7da0 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:42.269059 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79518ef-5761-4f9f-bfd0-60180ced7da0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:42.270204 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79518ef-5761-4f9f-bfd0-60180ced7da0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:42.271654 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79518ef-5761-4f9f-bfd0-60180ced7da0 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:42.273283 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 563/1109] 10.222.0.64 () {56 vars in 1108 bytes} [Tue Aug 2 09:47:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:47:42.279707 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2599862-5b87-4276-846e-c7716f2ed84b None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:42.297155 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:42.379702 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2599862-5b87-4276-846e-c7716f2ed84b None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:42.413170 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c0095f-9555-4ad2-9a4f-950bdb2aaf07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:42.413724 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c0095f-9555-4ad2-9a4f-950bdb2aaf07 None admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:42.414008 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c0095f-9555-4ad2-9a4f-950bdb2aaf07 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:42.414284 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c0095f-9555-4ad2-9a4f-950bdb2aaf07 None admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:42.414965 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c0095f-9555-4ad2-9a4f-950bdb2aaf07 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:42.416428 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c0095f-9555-4ad2-9a4f-950bdb2aaf07 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:42.436921 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 547/1110] 10.222.0.64 () {60 vars in 1132 bytes} [Tue Aug 2 09:47:42 2022] POST /identity/v3/users => generated 286 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:47:44.442501 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2919ba7-cd8d-463e-a77a-eff54ccbcb6f None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:44.443062 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2919ba7-cd8d-463e-a77a-eff54ccbcb6f None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:44.443062 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2919ba7-cd8d-463e-a77a-eff54ccbcb6f None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:44.444412 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 564/1111] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:47:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:47:44.449683 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1246ad-f5d9-4e61-abaf-f3a17cf57b8f None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:44.450086 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1246ad-f5d9-4e61-abaf-f3a17cf57b8f None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:44.450243 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1246ad-f5d9-4e61-abaf-f3a17cf57b8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:44.469721 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-9a1246ad-f5d9-4e61-abaf-f3a17cf57b8f None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:47:44.470962 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-9a1246ad-f5d9-4e61-abaf-f3a17cf57b8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:47:44.536330 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 548/1112] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:47:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 09:47:44.559035 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2919ba7-cd8d-463e-a77a-eff54ccbcb6f None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:44.648502 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:44.657982 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2919ba7-cd8d-463e-a77a-eff54ccbcb6f None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:44.690229 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f19245f-4801-4b96-a669-bd79298832ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:44.690853 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f19245f-4801-4b96-a669-bd79298832ba None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:44.691137 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f19245f-4801-4b96-a669-bd79298832ba None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:44.691411 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f19245f-4801-4b96-a669-bd79298832ba None admin] PATH_INFO: `/v3/roles/06dfd237063c42a4bff1ad8ca29c07b0` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:44.692899 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f19245f-4801-4b96-a669-bd79298832ba None admin] RBAC: Authorizing `identity:get_role(role_id=06dfd237063c42a4bff1ad8ca29c07b0)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:44.694214 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f19245f-4801-4b96-a669-bd79298832ba None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:44.695544 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 565/1113] 10.222.0.64 () {56 vars in 1177 bytes} [Tue Aug 2 09:47:44 2022] GET /identity/v3/roles/06dfd237063c42a4bff1ad8ca29c07b0 => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:47:44.702039 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1246ad-f5d9-4e61-abaf-f3a17cf57b8f None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:44.704174 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:44.797711 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1246ad-f5d9-4e61-abaf-f3a17cf57b8f None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:44.830821 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ae77a2-99be-4a1d-9494-f263466bea44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:44.831451 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ae77a2-99be-4a1d-9494-f263466bea44 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:44.831731 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ae77a2-99be-4a1d-9494-f263466bea44 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:44.832000 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ae77a2-99be-4a1d-9494-f263466bea44 None admin] PATH_INFO: `/v3/users/f7c4d604613b468194c9a958cc167044` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:44.844749 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ae77a2-99be-4a1d-9494-f263466bea44 None admin] RBAC: Authorizing `identity:get_user(user_id=f7c4d604613b468194c9a958cc167044)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:44.846184 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ae77a2-99be-4a1d-9494-f263466bea44 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:44.847822 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 549/1114] 10.222.0.64 () {56 vars in 1177 bytes} [Tue Aug 2 09:47:44 2022] GET /identity/v3/users/f7c4d604613b468194c9a958cc167044 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:47:44.853787 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f19245f-4801-4b96-a669-bd79298832ba None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:44.953067 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f19245f-4801-4b96-a669-bd79298832ba None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:44.985483 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2e18e5-0960-4ba1-aed0-b201e9201a58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:44.986086 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2e18e5-0960-4ba1-aed0-b201e9201a58 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:44.986367 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2e18e5-0960-4ba1-aed0-b201e9201a58 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:44.986640 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2e18e5-0960-4ba1-aed0-b201e9201a58 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:45.013176 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:45.051370 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2e18e5-0960-4ba1-aed0-b201e9201a58 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:45.052846 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2e18e5-0960-4ba1-aed0-b201e9201a58 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:45.147978 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 566/1115] 10.222.0.64 () {58 vars in 1282 bytes} [Tue Aug 2 09:47:44 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 02 09:47:45.153874 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ae77a2-99be-4a1d-9494-f263466bea44 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:45.211346 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:45.250674 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ae77a2-99be-4a1d-9494-f263466bea44 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:45.283615 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097cd7e5-6a05-4bae-b86d-0ff9402b39f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:45.284266 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097cd7e5-6a05-4bae-b86d-0ff9402b39f0 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:45.284550 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097cd7e5-6a05-4bae-b86d-0ff9402b39f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:45.284822 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097cd7e5-6a05-4bae-b86d-0ff9402b39f0 None admin] PATH_INFO: `/v3/projects/e502fb4fefa64be9b2e40bdec3c99876` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:45.285954 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097cd7e5-6a05-4bae-b86d-0ff9402b39f0 None admin] RBAC: Authorizing `identity:get_project(project_id=e502fb4fefa64be9b2e40bdec3c99876)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:45.287364 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097cd7e5-6a05-4bae-b86d-0ff9402b39f0 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:45.289035 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 550/1116] 10.222.0.64 () {56 vars in 1186 bytes} [Tue Aug 2 09:47:45 2022] GET /identity/v3/projects/e502fb4fefa64be9b2e40bdec3c99876 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:47:45.295157 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2e18e5-0960-4ba1-aed0-b201e9201a58 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:45.352142 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:45.393905 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2e18e5-0960-4ba1-aed0-b201e9201a58 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:45.427401 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1224cad9-8894-43d6-bae0-c66f66402244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:45.428013 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1224cad9-8894-43d6-bae0-c66f66402244 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:45.428326 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1224cad9-8894-43d6-bae0-c66f66402244 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:45.428606 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1224cad9-8894-43d6-bae0-c66f66402244 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:45.429398 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1224cad9-8894-43d6-bae0-c66f66402244 None admin] RBAC: Adding query filter params (role.id=06dfd237063c42a4bff1ad8ca29c07b0, scope.project.id=e502fb4fefa64be9b2e40bdec3c99876, user.id=f7c4d604613b468194c9a958cc167044) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:47:45.429712 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1224cad9-8894-43d6-bae0-c66f66402244 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:45.431138 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1224cad9-8894-43d6-bae0-c66f66402244 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:45.436449 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 567/1117] 10.222.0.64 () {56 vars in 1374 bytes} [Tue Aug 2 09:47:45 2022] GET /identity/v3/role_assignments?user.id=f7c4d604613b468194c9a958cc167044&scope.project.id=e502fb4fefa64be9b2e40bdec3c99876&role.id=06dfd237063c42a4bff1ad8ca29c07b0 => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:47:47.387411 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c905bab-52a9-4e35-a45b-2b4b6ebe2511 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:47.387948 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c905bab-52a9-4e35-a45b-2b4b6ebe2511 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:47.387948 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c905bab-52a9-4e35-a45b-2b4b6ebe2511 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:47.389354 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 551/1118] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:47:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:47:47.394537 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba56e78-eeb4-4b69-8f0a-d28e5def6d35 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:47.394987 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba56e78-eeb4-4b69-8f0a-d28e5def6d35 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:47.395081 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba56e78-eeb4-4b69-8f0a-d28e5def6d35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:47.412693 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-bba56e78-eeb4-4b69-8f0a-d28e5def6d35 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:47:47.413874 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-bba56e78-eeb4-4b69-8f0a-d28e5def6d35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:47:47.478957 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 568/1119] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:47:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 09:47:47.502170 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c905bab-52a9-4e35-a45b-2b4b6ebe2511 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:47.602654 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c905bab-52a9-4e35-a45b-2b4b6ebe2511 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:47.628468 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:47.636758 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b1a869-06e6-414d-92f2-5c5867ad9a17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:47.637395 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b1a869-06e6-414d-92f2-5c5867ad9a17 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:47.637730 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b1a869-06e6-414d-92f2-5c5867ad9a17 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:47.637963 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b1a869-06e6-414d-92f2-5c5867ad9a17 None admin] PATH_INFO: `/v3/roles/06dfd237063c42a4bff1ad8ca29c07b0` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:47.639441 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b1a869-06e6-414d-92f2-5c5867ad9a17 None admin] RBAC: Authorizing `identity:get_role(role_id=06dfd237063c42a4bff1ad8ca29c07b0)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:47.640851 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b1a869-06e6-414d-92f2-5c5867ad9a17 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:47.642200 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 552/1120] 10.222.0.64 () {56 vars in 1177 bytes} [Tue Aug 2 09:47:47 2022] GET /identity/v3/roles/06dfd237063c42a4bff1ad8ca29c07b0 => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:47:47.648138 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba56e78-eeb4-4b69-8f0a-d28e5def6d35 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:47.705104 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:47.748775 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba56e78-eeb4-4b69-8f0a-d28e5def6d35 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:47.781787 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630b97b0-ff63-4fd5-a61b-af8dd75e2fdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:47.782425 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630b97b0-ff63-4fd5-a61b-af8dd75e2fdc None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:47.782710 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630b97b0-ff63-4fd5-a61b-af8dd75e2fdc None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:47.782987 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630b97b0-ff63-4fd5-a61b-af8dd75e2fdc None admin] PATH_INFO: `/v3/users/f7c4d604613b468194c9a958cc167044` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:47.784133 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630b97b0-ff63-4fd5-a61b-af8dd75e2fdc None admin] RBAC: Authorizing `identity:get_user(user_id=f7c4d604613b468194c9a958cc167044)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:47.785590 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630b97b0-ff63-4fd5-a61b-af8dd75e2fdc None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:47.787204 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 569/1121] 10.222.0.64 () {56 vars in 1177 bytes} [Tue Aug 2 09:47:47 2022] GET /identity/v3/users/f7c4d604613b468194c9a958cc167044 => generated 286 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:47:47.793008 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b1a869-06e6-414d-92f2-5c5867ad9a17 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:47.893273 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b1a869-06e6-414d-92f2-5c5867ad9a17 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:47.926434 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117dcc5f-c4a3-4e61-975b-e7d4ba919630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:47.927071 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117dcc5f-c4a3-4e61-975b-e7d4ba919630 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:47.927359 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117dcc5f-c4a3-4e61-975b-e7d4ba919630 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:47.927634 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117dcc5f-c4a3-4e61-975b-e7d4ba919630 None admin] PATH_INFO: `/v3/projects/e502fb4fefa64be9b2e40bdec3c99876` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:47.928820 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117dcc5f-c4a3-4e61-975b-e7d4ba919630 None admin] RBAC: Authorizing `identity:get_project(project_id=e502fb4fefa64be9b2e40bdec3c99876)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:47.930247 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117dcc5f-c4a3-4e61-975b-e7d4ba919630 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:47.931922 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 553/1122] 10.222.0.64 () {56 vars in 1186 bytes} [Tue Aug 2 09:47:47 2022] GET /identity/v3/projects/e502fb4fefa64be9b2e40bdec3c99876 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:47:47.938896 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630b97b0-ff63-4fd5-a61b-af8dd75e2fdc None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:48.037850 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630b97b0-ff63-4fd5-a61b-af8dd75e2fdc None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:48.060945 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:48.073613 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b12090-7401-4e90-9572-228dcd4404dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:48.074282 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b12090-7401-4e90-9572-228dcd4404dd None admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:48.074567 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b12090-7401-4e90-9572-228dcd4404dd None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:48.074844 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b12090-7401-4e90-9572-228dcd4404dd None admin] PATH_INFO: `/v3/projects/e502fb4fefa64be9b2e40bdec3c99876/users/f7c4d604613b468194c9a958cc167044/roles/06dfd237063c42a4bff1ad8ca29c07b0` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:48.077110 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b12090-7401-4e90-9572-228dcd4404dd None admin] RBAC: Authorizing `identity:create_grant(project_id=e502fb4fefa64be9b2e40bdec3c99876, user_id=f7c4d604613b468194c9a958cc167044, role_id=06dfd237063c42a4bff1ad8ca29c07b0)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:48.078528 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b12090-7401-4e90-9572-228dcd4404dd None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:48.093948 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 570/1123] 10.222.0.64 () {58 vars in 1439 bytes} [Tue Aug 2 09:47:47 2022] PUT /identity/v3/projects/e502fb4fefa64be9b2e40bdec3c99876/users/f7c4d604613b468194c9a958cc167044/roles/06dfd237063c42a4bff1ad8ca29c07b0 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:47:50.080799 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32665dc0-3a19-42c0-b6f3-4e1940a7d866 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:50.081356 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32665dc0-3a19-42c0-b6f3-4e1940a7d866 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:50.081356 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32665dc0-3a19-42c0-b6f3-4e1940a7d866 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:50.082746 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 554/1124] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:47:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:47:50.088282 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2a103b-9685-49d7-81c6-05196de655ca None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:50.088886 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2a103b-9685-49d7-81c6-05196de655ca None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:50.088886 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2a103b-9685-49d7-81c6-05196de655ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:50.106959 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-fb2a103b-9685-49d7-81c6-05196de655ca None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:47:50.108158 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-fb2a103b-9685-49d7-81c6-05196de655ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:47:50.174130 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 571/1125] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:47:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 02 09:47:50.197691 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32665dc0-3a19-42c0-b6f3-4e1940a7d866 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:50.262508 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:50.302055 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32665dc0-3a19-42c0-b6f3-4e1940a7d866 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:50.336920 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7a1048-381c-439a-bd00-9825626cb5c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:50.337552 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7a1048-381c-439a-bd00-9825626cb5c4 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:50.337831 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7a1048-381c-439a-bd00-9825626cb5c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:50.338101 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7a1048-381c-439a-bd00-9825626cb5c4 None admin] PATH_INFO: `/v3/roles/06dfd237063c42a4bff1ad8ca29c07b0` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:50.339530 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7a1048-381c-439a-bd00-9825626cb5c4 None admin] RBAC: Authorizing `identity:get_role(role_id=06dfd237063c42a4bff1ad8ca29c07b0)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:50.340957 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7a1048-381c-439a-bd00-9825626cb5c4 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:50.343122 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 555/1126] 10.222.0.64 () {56 vars in 1177 bytes} [Tue Aug 2 09:47:50 2022] GET /identity/v3/roles/06dfd237063c42a4bff1ad8ca29c07b0 => generated 223 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:47:50.349316 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2a103b-9685-49d7-81c6-05196de655ca None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:50.449723 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2a103b-9685-49d7-81c6-05196de655ca None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:50.463099 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:50.485783 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f42c614-1a26-4041-afd8-8cb73c9cced5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:50.486428 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f42c614-1a26-4041-afd8-8cb73c9cced5 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:50.486712 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f42c614-1a26-4041-afd8-8cb73c9cced5 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:50.486985 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f42c614-1a26-4041-afd8-8cb73c9cced5 None admin] PATH_INFO: `/v3/users/f7c4d604613b468194c9a958cc167044` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:50.488193 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f42c614-1a26-4041-afd8-8cb73c9cced5 None admin] RBAC: Authorizing `identity:get_user(user_id=f7c4d604613b468194c9a958cc167044)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:50.489663 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f42c614-1a26-4041-afd8-8cb73c9cced5 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:50.491266 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 572/1127] 10.222.0.64 () {56 vars in 1177 bytes} [Tue Aug 2 09:47:50 2022] GET /identity/v3/users/f7c4d604613b468194c9a958cc167044 => generated 286 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:47:50.497126 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7a1048-381c-439a-bd00-9825626cb5c4 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:50.598861 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7a1048-381c-439a-bd00-9825626cb5c4 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:50.631279 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:50.636586 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68584f30-7a2c-4228-ba14-506b7e6718fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:50.637203 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68584f30-7a2c-4228-ba14-506b7e6718fa None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:50.637483 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68584f30-7a2c-4228-ba14-506b7e6718fa None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:50.637754 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68584f30-7a2c-4228-ba14-506b7e6718fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:50.705922 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68584f30-7a2c-4228-ba14-506b7e6718fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:50.707288 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68584f30-7a2c-4228-ba14-506b7e6718fa None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:50.811347 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 556/1128] 10.222.0.64 () {58 vars in 1282 bytes} [Tue Aug 2 09:47:50 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 02 09:47:50.817478 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f42c614-1a26-4041-afd8-8cb73c9cced5 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:50.917066 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f42c614-1a26-4041-afd8-8cb73c9cced5 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:50.951734 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbefd809-90bb-4b6d-bc02-b1385a6ce168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:50.952398 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbefd809-90bb-4b6d-bc02-b1385a6ce168 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:50.952687 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbefd809-90bb-4b6d-bc02-b1385a6ce168 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:50.952966 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbefd809-90bb-4b6d-bc02-b1385a6ce168 None admin] PATH_INFO: `/v3/projects/e502fb4fefa64be9b2e40bdec3c99876` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:50.954126 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbefd809-90bb-4b6d-bc02-b1385a6ce168 None admin] RBAC: Authorizing `identity:get_project(project_id=e502fb4fefa64be9b2e40bdec3c99876)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:50.955574 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbefd809-90bb-4b6d-bc02-b1385a6ce168 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:50.957323 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 573/1129] 10.222.0.64 () {56 vars in 1186 bytes} [Tue Aug 2 09:47:50 2022] GET /identity/v3/projects/e502fb4fefa64be9b2e40bdec3c99876 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:47:50.963472 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68584f30-7a2c-4228-ba14-506b7e6718fa None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:50.993659 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:51.061197 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68584f30-7a2c-4228-ba14-506b7e6718fa None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:51.093277 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba701d44-58aa-4ced-9460-28a1ba7c2170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:51.093866 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba701d44-58aa-4ced-9460-28a1ba7c2170 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:51.094147 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba701d44-58aa-4ced-9460-28a1ba7c2170 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:51.094418 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba701d44-58aa-4ced-9460-28a1ba7c2170 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:51.095201 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba701d44-58aa-4ced-9460-28a1ba7c2170 None admin] RBAC: Adding query filter params (role.id=06dfd237063c42a4bff1ad8ca29c07b0, scope.project.id=e502fb4fefa64be9b2e40bdec3c99876, user.id=f7c4d604613b468194c9a958cc167044) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:47:51.095511 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba701d44-58aa-4ced-9460-28a1ba7c2170 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:51.096954 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba701d44-58aa-4ced-9460-28a1ba7c2170 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:51.102913 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 557/1130] 10.222.0.64 () {56 vars in 1374 bytes} [Tue Aug 2 09:47:50 2022] GET /identity/v3/role_assignments?user.id=f7c4d604613b468194c9a958cc167044&scope.project.id=e502fb4fefa64be9b2e40bdec3c99876&role.id=06dfd237063c42a4bff1ad8ca29c07b0 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:47:53.109541 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18dc11d-ac75-47c9-a4d2-1ccb84491406 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:53.110076 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18dc11d-ac75-47c9-a4d2-1ccb84491406 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:53.110076 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18dc11d-ac75-47c9-a4d2-1ccb84491406 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:53.111416 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 574/1131] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:47:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 09:47:53.116930 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3f11c2-05be-4087-bea8-6c9fff01c912 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:53.117522 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3f11c2-05be-4087-bea8-6c9fff01c912 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:53.117522 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3f11c2-05be-4087-bea8-6c9fff01c912 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:53.135383 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-5c3f11c2-05be-4087-bea8-6c9fff01c912 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:47:53.136616 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-5c3f11c2-05be-4087-bea8-6c9fff01c912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:47:53.208417 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 558/1132] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:47:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 09:47:53.232662 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18dc11d-ac75-47c9-a4d2-1ccb84491406 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:53.282184 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:53.335432 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18dc11d-ac75-47c9-a4d2-1ccb84491406 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:53.375117 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6c3830-a1ab-47a9-b54e-982498b91211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:53.375752 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6c3830-a1ab-47a9-b54e-982498b91211 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:53.376034 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6c3830-a1ab-47a9-b54e-982498b91211 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:53.376335 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6c3830-a1ab-47a9-b54e-982498b91211 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:53.377476 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6c3830-a1ab-47a9-b54e-982498b91211 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:53.378903 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6c3830-a1ab-47a9-b54e-982498b91211 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:53.380551 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 575/1133] 10.222.0.64 () {56 vars in 1108 bytes} [Tue Aug 2 09:47:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:47:53.386690 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3f11c2-05be-4087-bea8-6c9fff01c912 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:53.414135 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:53.491255 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3f11c2-05be-4087-bea8-6c9fff01c912 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:53.525113 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd399f8-6839-4d55-982e-5ec5b2daa2bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:53.525725 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd399f8-6839-4d55-982e-5ec5b2daa2bf None admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:53.526008 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd399f8-6839-4d55-982e-5ec5b2daa2bf None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:53.526282 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd399f8-6839-4d55-982e-5ec5b2daa2bf None admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:53.526993 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd399f8-6839-4d55-982e-5ec5b2daa2bf None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:53.528436 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd399f8-6839-4d55-982e-5ec5b2daa2bf None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:53.555378 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 559/1134] 10.222.0.64 () {60 vars in 1141 bytes} [Tue Aug 2 09:47:53 2022] POST /identity/v3/projects => generated 311 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:47:55.595355 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7921cb1c-b4a3-4b1f-b3b9-ca449e7d4c11 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:55.595974 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7921cb1c-b4a3-4b1f-b3b9-ca449e7d4c11 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:55.595974 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7921cb1c-b4a3-4b1f-b3b9-ca449e7d4c11 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:55.597316 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 576/1135] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:47:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:47:55.602937 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb334a7d-3ad0-43c3-9b05-68c0c773dfeb None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:55.603359 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb334a7d-3ad0-43c3-9b05-68c0c773dfeb None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:55.603512 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb334a7d-3ad0-43c3-9b05-68c0c773dfeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:55.622477 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-bb334a7d-3ad0-43c3-9b05-68c0c773dfeb None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:47:55.624227 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-bb334a7d-3ad0-43c3-9b05-68c0c773dfeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:47:55.704592 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 560/1136] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:47:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 09:47:55.729613 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7921cb1c-b4a3-4b1f-b3b9-ca449e7d4c11 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:55.836128 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7921cb1c-b4a3-4b1f-b3b9-ca449e7d4c11 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:55.854151 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:55.873116 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0dd65e-0563-4acf-900e-52bd7d606171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:55.873677 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0dd65e-0563-4acf-900e-52bd7d606171 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:55.873959 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0dd65e-0563-4acf-900e-52bd7d606171 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:55.874235 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0dd65e-0563-4acf-900e-52bd7d606171 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:55.875400 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0dd65e-0563-4acf-900e-52bd7d606171 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:55.876882 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0dd65e-0563-4acf-900e-52bd7d606171 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:55.878516 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 577/1137] 10.222.0.64 () {56 vars in 1108 bytes} [Tue Aug 2 09:47:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:47:55.884346 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb334a7d-3ad0-43c3-9b05-68c0c773dfeb None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:55.894514 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:55.988492 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb334a7d-3ad0-43c3-9b05-68c0c773dfeb None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:56.021716 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535a34b5-179a-4e42-9fdd-c779fe49570c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:56.022332 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535a34b5-179a-4e42-9fdd-c779fe49570c None admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:56.022611 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535a34b5-179a-4e42-9fdd-c779fe49570c None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:56.022883 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535a34b5-179a-4e42-9fdd-c779fe49570c None admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:56.023575 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535a34b5-179a-4e42-9fdd-c779fe49570c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:56.025037 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535a34b5-179a-4e42-9fdd-c779fe49570c None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:56.047773 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 561/1138] 10.222.0.64 () {60 vars in 1132 bytes} [Tue Aug 2 09:47:55 2022] POST /identity/v3/users => generated 286 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:47:58.104469 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d6beca-81d0-46df-982a-2cabe8aa9767 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:58.105032 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d6beca-81d0-46df-982a-2cabe8aa9767 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:58.105032 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d6beca-81d0-46df-982a-2cabe8aa9767 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:58.106353 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 578/1139] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:47:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:47:58.111686 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7257b60a-4ae3-470e-be64-f14b0d424073 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:58.112149 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7257b60a-4ae3-470e-be64-f14b0d424073 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:58.112283 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7257b60a-4ae3-470e-be64-f14b0d424073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:58.130800 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-7257b60a-4ae3-470e-be64-f14b0d424073 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:47:58.131992 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-7257b60a-4ae3-470e-be64-f14b0d424073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:47:58.191455 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:58.201129 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 562/1140] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:47:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 09:47:58.224092 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d6beca-81d0-46df-982a-2cabe8aa9767 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:58.326486 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d6beca-81d0-46df-982a-2cabe8aa9767 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:58.362085 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc8fc3e-c19b-4c7d-b1a8-f045d44becec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:58.362730 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc8fc3e-c19b-4c7d-b1a8-f045d44becec None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:58.363016 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc8fc3e-c19b-4c7d-b1a8-f045d44becec None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:58.363310 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc8fc3e-c19b-4c7d-b1a8-f045d44becec None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:58.373907 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc8fc3e-c19b-4c7d-b1a8-f045d44becec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 09:47:58.378759 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-cbc8fc3e-c19b-4c7d-b1a8-f045d44becec None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 09:47:58.378759 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:47:58.378759 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:47:58.378759 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:47:58.378759 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:47:58.378759 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:47:58.378759 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:47:58.378759 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:47:58.378759 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:47:58.378759 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:47:58.378759 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:47:58.378759 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:47:58.378759 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 09:47:58.378759 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 09:47:58.378759 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 09:47:58.378759 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 09:47:58.379859 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 09:47:58.379859 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 09:47:58.379859 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 09:47:58.379859 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 09:47:58.379859 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:47:58.379859 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:47:58.379859 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:47:58.379859 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:47:58.379859 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:47:58.379859 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:47:58.379859 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:47:58.379859 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:47:58.379859 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:47:58.379859 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:47:58.379859 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:47:58.379859 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:47:58.380962 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:47:58.380962 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:47:58.380962 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:47:58.380962 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:47:58.380962 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 09:47:58.380962 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 09:47:58.380962 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 09:47:58.380962 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 09:47:58.380962 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 09:47:58.380962 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 09:47:58.380962 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 09:47:58.380962 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:47:58.380962 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 579/1141] 10.222.0.64 () {56 vars in 1096 bytes} [Tue Aug 2 09:47:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:47:58.386150 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7257b60a-4ae3-470e-be64-f14b0d424073 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:58.486419 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7257b60a-4ae3-470e-be64-f14b0d424073 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:58.520622 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0386a9d6-ad13-481e-aa77-85f80d84f396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:58.521239 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0386a9d6-ad13-481e-aa77-85f80d84f396 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:58.521519 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0386a9d6-ad13-481e-aa77-85f80d84f396 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:58.521788 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0386a9d6-ad13-481e-aa77-85f80d84f396 None admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:58.522506 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0386a9d6-ad13-481e-aa77-85f80d84f396 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:47:58.522808 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0386a9d6-ad13-481e-aa77-85f80d84f396 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:58.524169 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0386a9d6-ad13-481e-aa77-85f80d84f396 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:58.532652 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 563/1142] 10.222.0.64 () {56 vars in 1099 bytes} [Tue Aug 2 09:47:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:47:58.538570 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc8fc3e-c19b-4c7d-b1a8-f045d44becec None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:58.587436 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:58.642712 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc8fc3e-c19b-4c7d-b1a8-f045d44becec None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:58.678833 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f396b3e9-682b-425a-a9a8-a579559673ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:58.679479 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f396b3e9-682b-425a-a9a8-a579559673ea None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:58.679766 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f396b3e9-682b-425a-a9a8-a579559673ea None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:58.680047 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f396b3e9-682b-425a-a9a8-a579559673ea None admin] PATH_INFO: `/v3/users/bcb2475bfa1e4d1f8283e7628467fa49` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:58.695781 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f396b3e9-682b-425a-a9a8-a579559673ea None admin] RBAC: Authorizing `identity:get_user(user_id=bcb2475bfa1e4d1f8283e7628467fa49)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:58.697302 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f396b3e9-682b-425a-a9a8-a579559673ea None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:58.699035 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 580/1143] 10.222.0.64 () {56 vars in 1177 bytes} [Tue Aug 2 09:47:58 2022] GET /identity/v3/users/bcb2475bfa1e4d1f8283e7628467fa49 => generated 286 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:47:58.704969 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0386a9d6-ad13-481e-aa77-85f80d84f396 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:58.762314 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:58.806984 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0386a9d6-ad13-481e-aa77-85f80d84f396 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:58.844568 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9b0407-838f-4f59-bf4c-b4dd948a2884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:58.845201 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9b0407-838f-4f59-bf4c-b4dd948a2884 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:58.845485 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9b0407-838f-4f59-bf4c-b4dd948a2884 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:58.845758 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9b0407-838f-4f59-bf4c-b4dd948a2884 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:58.924860 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9b0407-838f-4f59-bf4c-b4dd948a2884 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:58.926326 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9b0407-838f-4f59-bf4c-b4dd948a2884 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:58.982577 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:59.030122 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 564/1144] 10.222.0.64 () {58 vars in 1282 bytes} [Tue Aug 2 09:47:58 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 02 09:47:59.035972 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f396b3e9-682b-425a-a9a8-a579559673ea None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:59.136613 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f396b3e9-682b-425a-a9a8-a579559673ea None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:59.138185 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:47:59.173225 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3f131b-b321-4e65-a921-d9808103f5f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:59.173879 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3f131b-b321-4e65-a921-d9808103f5f7 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:59.174162 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3f131b-b321-4e65-a921-d9808103f5f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:59.174441 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3f131b-b321-4e65-a921-d9808103f5f7 None admin] PATH_INFO: `/v3/projects/b9bcf94506f440988dd470bf1f0a56b3` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:59.175658 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3f131b-b321-4e65-a921-d9808103f5f7 None admin] RBAC: Authorizing `identity:get_project(project_id=b9bcf94506f440988dd470bf1f0a56b3)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:59.177095 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3f131b-b321-4e65-a921-d9808103f5f7 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:59.178789 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 581/1145] 10.222.0.64 () {56 vars in 1186 bytes} [Tue Aug 2 09:47:59 2022] GET /identity/v3/projects/b9bcf94506f440988dd470bf1f0a56b3 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:47:59.185256 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9b0407-838f-4f59-bf4c-b4dd948a2884 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:47:59.296274 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9b0407-838f-4f59-bf4c-b4dd948a2884 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:47:59.330420 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1a0881-e681-4666-aec7-a4c7ee7aa2b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:47:59.331056 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1a0881-e681-4666-aec7-a4c7ee7aa2b2 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:47:59.331335 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1a0881-e681-4666-aec7-a4c7ee7aa2b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:47:59.331601 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1a0881-e681-4666-aec7-a4c7ee7aa2b2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:47:59.332423 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1a0881-e681-4666-aec7-a4c7ee7aa2b2 None admin] RBAC: Adding query filter params (role.id=1000cf318fa14915b8091599eaead5de, scope.project.id=b9bcf94506f440988dd470bf1f0a56b3, user.id=bcb2475bfa1e4d1f8283e7628467fa49) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:47:59.332732 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1a0881-e681-4666-aec7-a4c7ee7aa2b2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:47:59.334157 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1a0881-e681-4666-aec7-a4c7ee7aa2b2 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:47:59.340016 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 565/1146] 10.222.0.64 () {56 vars in 1374 bytes} [Tue Aug 2 09:47:59 2022] GET /identity/v3/role_assignments?user.id=bcb2475bfa1e4d1f8283e7628467fa49&scope.project.id=b9bcf94506f440988dd470bf1f0a56b3&role.id=1000cf318fa14915b8091599eaead5de => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:48:01.331237 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f9a6f9-9c90-4e33-b995-2319bc984256 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:01.331775 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f9a6f9-9c90-4e33-b995-2319bc984256 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:01.331775 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f9a6f9-9c90-4e33-b995-2319bc984256 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:01.336206 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 582/1147] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:48:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:48:01.338700 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb07904-8261-4310-aa4b-18487e60a552 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:01.339153 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb07904-8261-4310-aa4b-18487e60a552 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:01.339246 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb07904-8261-4310-aa4b-18487e60a552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:01.358730 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-9eb07904-8261-4310-aa4b-18487e60a552 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:48:01.359935 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-9eb07904-8261-4310-aa4b-18487e60a552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:48:01.374608 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:48:01.433308 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 566/1148] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:48:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 09:48:01.456500 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f9a6f9-9c90-4e33-b995-2319bc984256 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:01.556785 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f9a6f9-9c90-4e33-b995-2319bc984256 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:01.591836 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bd5f61-002c-404d-9828-2001661d4e83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:01.592441 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bd5f61-002c-404d-9828-2001661d4e83 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:01.592725 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bd5f61-002c-404d-9828-2001661d4e83 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:01.592996 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bd5f61-002c-404d-9828-2001661d4e83 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:01.603091 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bd5f61-002c-404d-9828-2001661d4e83 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 09:48:01.608722 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-11bd5f61-002c-404d-9828-2001661d4e83 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 09:48:01.608722 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:48:01.608722 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:48:01.608722 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:48:01.608722 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:48:01.608722 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:48:01.608722 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:48:01.608722 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:48:01.608722 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:48:01.608722 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:48:01.608722 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:48:01.608722 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:48:01.608722 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 09:48:01.608722 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 09:48:01.608722 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 09:48:01.608722 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 09:48:01.609801 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 09:48:01.609801 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 09:48:01.609801 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 09:48:01.609801 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 09:48:01.609801 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:48:01.609801 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:48:01.609801 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:48:01.609801 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:48:01.609801 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:48:01.609801 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:48:01.609801 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:48:01.609801 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:48:01.609801 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:48:01.609801 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:48:01.609801 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:48:01.609801 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:48:01.610809 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:48:01.610809 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:48:01.610809 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:48:01.610809 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:48:01.610809 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 09:48:01.610809 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 09:48:01.610809 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 09:48:01.610809 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 09:48:01.610809 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 09:48:01.610809 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 09:48:01.610809 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 09:48:01.610809 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:48:01.610809 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 583/1149] 10.222.0.64 () {56 vars in 1096 bytes} [Tue Aug 2 09:48:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:48:01.615927 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb07904-8261-4310-aa4b-18487e60a552 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:01.716528 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb07904-8261-4310-aa4b-18487e60a552 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:01.756113 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed5f65d-cc69-4257-856f-c5bb39c0402d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:01.756766 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed5f65d-cc69-4257-856f-c5bb39c0402d None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:01.757056 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed5f65d-cc69-4257-856f-c5bb39c0402d None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:01.757333 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed5f65d-cc69-4257-856f-c5bb39c0402d None admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:01.758064 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed5f65d-cc69-4257-856f-c5bb39c0402d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:48:01.758372 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed5f65d-cc69-4257-856f-c5bb39c0402d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:01.759748 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed5f65d-cc69-4257-856f-c5bb39c0402d None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:01.768661 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 567/1150] 10.222.0.64 () {56 vars in 1099 bytes} [Tue Aug 2 09:48:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:48:01.774604 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bd5f61-002c-404d-9828-2001661d4e83 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:01.805882 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:48:01.882066 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bd5f61-002c-404d-9828-2001661d4e83 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:01.916010 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640c2cfa-33a5-4e7e-826d-d18564200b75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:01.916675 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640c2cfa-33a5-4e7e-826d-d18564200b75 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:01.916960 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640c2cfa-33a5-4e7e-826d-d18564200b75 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:01.917235 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640c2cfa-33a5-4e7e-826d-d18564200b75 None admin] PATH_INFO: `/v3/users/bcb2475bfa1e4d1f8283e7628467fa49` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:01.918388 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640c2cfa-33a5-4e7e-826d-d18564200b75 None admin] RBAC: Authorizing `identity:get_user(user_id=bcb2475bfa1e4d1f8283e7628467fa49)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:01.920518 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640c2cfa-33a5-4e7e-826d-d18564200b75 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:01.922106 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 584/1151] 10.222.0.64 () {56 vars in 1177 bytes} [Tue Aug 2 09:48:01 2022] GET /identity/v3/users/bcb2475bfa1e4d1f8283e7628467fa49 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:48:01.927840 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed5f65d-cc69-4257-856f-c5bb39c0402d None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:01.942637 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:48:02.024917 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed5f65d-cc69-4257-856f-c5bb39c0402d None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:02.057921 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c10d331-fa99-4295-bda1-e7aea3428fdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:02.058547 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c10d331-fa99-4295-bda1-e7aea3428fdf None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:02.058829 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c10d331-fa99-4295-bda1-e7aea3428fdf None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:02.059100 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c10d331-fa99-4295-bda1-e7aea3428fdf None admin] PATH_INFO: `/v3/projects/b9bcf94506f440988dd470bf1f0a56b3` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:02.060342 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c10d331-fa99-4295-bda1-e7aea3428fdf None admin] RBAC: Authorizing `identity:get_project(project_id=b9bcf94506f440988dd470bf1f0a56b3)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:02.061766 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c10d331-fa99-4295-bda1-e7aea3428fdf None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:02.063422 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 568/1152] 10.222.0.64 () {56 vars in 1186 bytes} [Tue Aug 2 09:48:01 2022] GET /identity/v3/projects/b9bcf94506f440988dd470bf1f0a56b3 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:48:02.069440 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640c2cfa-33a5-4e7e-826d-d18564200b75 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:02.162366 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:48:02.166528 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640c2cfa-33a5-4e7e-826d-d18564200b75 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:02.199875 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90458900-a735-431c-b7a7-8220861e6329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:02.200766 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90458900-a735-431c-b7a7-8220861e6329 None admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:02.201050 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90458900-a735-431c-b7a7-8220861e6329 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:02.201326 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90458900-a735-431c-b7a7-8220861e6329 None admin] PATH_INFO: `/v3/projects/b9bcf94506f440988dd470bf1f0a56b3/users/bcb2475bfa1e4d1f8283e7628467fa49/roles/1000cf318fa14915b8091599eaead5de` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:02.203240 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90458900-a735-431c-b7a7-8220861e6329 None admin] RBAC: Authorizing `identity:create_grant(project_id=b9bcf94506f440988dd470bf1f0a56b3, user_id=bcb2475bfa1e4d1f8283e7628467fa49, role_id=1000cf318fa14915b8091599eaead5de)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:02.204707 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90458900-a735-431c-b7a7-8220861e6329 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:02.218186 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 585/1153] 10.222.0.64 () {58 vars in 1439 bytes} [Tue Aug 2 09:48:02 2022] PUT /identity/v3/projects/b9bcf94506f440988dd470bf1f0a56b3/users/bcb2475bfa1e4d1f8283e7628467fa49/roles/1000cf318fa14915b8091599eaead5de => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:48:04.192677 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0186b76-1f13-425c-b714-81a149392e3a None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:04.193198 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0186b76-1f13-425c-b714-81a149392e3a None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:04.193198 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0186b76-1f13-425c-b714-81a149392e3a None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:04.194514 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 569/1154] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:48:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:48:04.199886 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3a4f93-d88b-4f48-a2b4-5eb054594fcc None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:04.200954 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3a4f93-d88b-4f48-a2b4-5eb054594fcc None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:04.201131 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3a4f93-d88b-4f48-a2b4-5eb054594fcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:04.218901 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-4f3a4f93-d88b-4f48-a2b4-5eb054594fcc None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:48:04.220101 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-4f3a4f93-d88b-4f48-a2b4-5eb054594fcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:48:04.288951 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 586/1155] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:48:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 09:48:04.312424 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0186b76-1f13-425c-b714-81a149392e3a None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:04.406620 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:48:04.427067 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0186b76-1f13-425c-b714-81a149392e3a None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:04.461696 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7dfc78-eba7-480e-9ae0-108b65c5a55e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:04.462261 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7dfc78-eba7-480e-9ae0-108b65c5a55e None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:04.462545 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7dfc78-eba7-480e-9ae0-108b65c5a55e None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:04.462818 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7dfc78-eba7-480e-9ae0-108b65c5a55e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:04.473720 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7dfc78-eba7-480e-9ae0-108b65c5a55e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 09:48:04.478560 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-6c7dfc78-eba7-480e-9ae0-108b65c5a55e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 09:48:04.478560 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:48:04.478560 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:48:04.478560 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:48:04.478560 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:48:04.478560 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:48:04.478560 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:48:04.478560 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:48:04.478560 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:48:04.478560 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:48:04.478560 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:48:04.478560 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:48:04.478560 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 09:48:04.478560 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 09:48:04.478560 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 09:48:04.478560 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 09:48:04.479685 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 09:48:04.479685 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 09:48:04.479685 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 09:48:04.479685 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 09:48:04.479685 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:48:04.479685 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:48:04.479685 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:48:04.479685 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:48:04.479685 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:48:04.479685 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:48:04.479685 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:48:04.479685 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:48:04.479685 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:48:04.479685 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:48:04.479685 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:48:04.479685 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:48:04.480649 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:48:04.480649 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:48:04.480649 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:48:04.480649 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:48:04.480649 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 09:48:04.480649 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 09:48:04.480649 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 09:48:04.480649 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 09:48:04.480649 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 09:48:04.480649 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 09:48:04.480649 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 09:48:04.480649 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:48:04.480649 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 570/1156] 10.222.0.64 () {56 vars in 1096 bytes} [Tue Aug 2 09:48:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:48:04.486740 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3a4f93-d88b-4f48-a2b4-5eb054594fcc None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:04.575776 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:48:04.589743 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3a4f93-d88b-4f48-a2b4-5eb054594fcc None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:04.626247 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27655fa1-0a09-4e81-bcf5-a9b526d66dc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:04.626926 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27655fa1-0a09-4e81-bcf5-a9b526d66dc4 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:04.627309 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27655fa1-0a09-4e81-bcf5-a9b526d66dc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:04.627533 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27655fa1-0a09-4e81-bcf5-a9b526d66dc4 None admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:04.628189 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27655fa1-0a09-4e81-bcf5-a9b526d66dc4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:48:04.628464 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27655fa1-0a09-4e81-bcf5-a9b526d66dc4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:04.629851 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27655fa1-0a09-4e81-bcf5-a9b526d66dc4 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:04.638335 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 587/1157] 10.222.0.64 () {56 vars in 1099 bytes} [Tue Aug 2 09:48:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:48:04.644145 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7dfc78-eba7-480e-9ae0-108b65c5a55e None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:04.757210 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7dfc78-eba7-480e-9ae0-108b65c5a55e None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:04.790344 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664aaaf6-242f-4a8e-a8d3-b3290a481679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:04.790989 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664aaaf6-242f-4a8e-a8d3-b3290a481679 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:04.791278 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664aaaf6-242f-4a8e-a8d3-b3290a481679 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:04.791558 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664aaaf6-242f-4a8e-a8d3-b3290a481679 None admin] PATH_INFO: `/v3/users/bcb2475bfa1e4d1f8283e7628467fa49` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:04.792765 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664aaaf6-242f-4a8e-a8d3-b3290a481679 None admin] RBAC: Authorizing `identity:get_user(user_id=bcb2475bfa1e4d1f8283e7628467fa49)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:04.794727 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664aaaf6-242f-4a8e-a8d3-b3290a481679 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:04.796609 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 571/1158] 10.222.0.64 () {56 vars in 1177 bytes} [Tue Aug 2 09:48:04 2022] GET /identity/v3/users/bcb2475bfa1e4d1f8283e7628467fa49 => generated 286 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:48:04.802209 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27655fa1-0a09-4e81-bcf5-a9b526d66dc4 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:04.918039 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27655fa1-0a09-4e81-bcf5-a9b526d66dc4 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:04.957269 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13257422-bbeb-4ebb-b4af-4de0b752d078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:04.957881 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13257422-bbeb-4ebb-b4af-4de0b752d078 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:04.958163 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13257422-bbeb-4ebb-b4af-4de0b752d078 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:04.958435 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13257422-bbeb-4ebb-b4af-4de0b752d078 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:04.979832 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:48:05.029800 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13257422-bbeb-4ebb-b4af-4de0b752d078 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:05.031167 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13257422-bbeb-4ebb-b4af-4de0b752d078 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:05.132332 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 588/1159] 10.222.0.64 () {58 vars in 1282 bytes} [Tue Aug 2 09:48:04 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 02 09:48:05.138230 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664aaaf6-242f-4a8e-a8d3-b3290a481679 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:05.150157 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:48:05.267091 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664aaaf6-242f-4a8e-a8d3-b3290a481679 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:05.304643 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8176f3ac-e8f4-42f4-a959-921fde498432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:05.305202 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8176f3ac-e8f4-42f4-a959-921fde498432 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:05.305484 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8176f3ac-e8f4-42f4-a959-921fde498432 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:05.305759 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8176f3ac-e8f4-42f4-a959-921fde498432 None admin] PATH_INFO: `/v3/projects/b9bcf94506f440988dd470bf1f0a56b3` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:05.306941 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8176f3ac-e8f4-42f4-a959-921fde498432 None admin] RBAC: Authorizing `identity:get_project(project_id=b9bcf94506f440988dd470bf1f0a56b3)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:05.308402 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8176f3ac-e8f4-42f4-a959-921fde498432 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:05.310901 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 572/1160] 10.222.0.64 () {56 vars in 1186 bytes} [Tue Aug 2 09:48:05 2022] GET /identity/v3/projects/b9bcf94506f440988dd470bf1f0a56b3 => generated 311 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:48:05.317261 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13257422-bbeb-4ebb-b4af-4de0b752d078 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:05.367181 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:48:05.417274 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13257422-bbeb-4ebb-b4af-4de0b752d078 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:05.457754 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d41bee-cfde-4bfc-9de0-3646fc860c54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:05.458372 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d41bee-cfde-4bfc-9de0-3646fc860c54 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:05.458655 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d41bee-cfde-4bfc-9de0-3646fc860c54 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:05.458932 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d41bee-cfde-4bfc-9de0-3646fc860c54 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:05.459726 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d41bee-cfde-4bfc-9de0-3646fc860c54 None admin] RBAC: Adding query filter params (role.id=1000cf318fa14915b8091599eaead5de, scope.project.id=b9bcf94506f440988dd470bf1f0a56b3, user.id=bcb2475bfa1e4d1f8283e7628467fa49) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:48:05.460037 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d41bee-cfde-4bfc-9de0-3646fc860c54 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:05.461516 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d41bee-cfde-4bfc-9de0-3646fc860c54 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:05.467716 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 589/1161] 10.222.0.64 () {56 vars in 1374 bytes} [Tue Aug 2 09:48:05 2022] GET /identity/v3/role_assignments?user.id=bcb2475bfa1e4d1f8283e7628467fa49&scope.project.id=b9bcf94506f440988dd470bf1f0a56b3&role.id=1000cf318fa14915b8091599eaead5de => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:48:07.495843 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e622b6-7d44-4971-965d-fe5e87749221 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:07.496407 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e622b6-7d44-4971-965d-fe5e87749221 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:07.496519 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e622b6-7d44-4971-965d-fe5e87749221 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:07.497813 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 573/1162] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:48:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:48:07.503190 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63efeec8-837c-44da-bfdf-fbeb9e5987f7 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:07.503902 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63efeec8-837c-44da-bfdf-fbeb9e5987f7 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:07.503902 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63efeec8-837c-44da-bfdf-fbeb9e5987f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:07.531099 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-63efeec8-837c-44da-bfdf-fbeb9e5987f7 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:48:07.532370 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-63efeec8-837c-44da-bfdf-fbeb9e5987f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:48:07.611185 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 590/1163] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:48:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 02 09:48:07.633003 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e622b6-7d44-4971-965d-fe5e87749221 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:07.725325 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:48:07.751518 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e622b6-7d44-4971-965d-fe5e87749221 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:07.788552 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4b1192-4aed-4a55-80fd-f199fb102cd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:07.789174 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4b1192-4aed-4a55-80fd-f199fb102cd3 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:07.789457 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4b1192-4aed-4a55-80fd-f199fb102cd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:07.789731 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4b1192-4aed-4a55-80fd-f199fb102cd3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:07.862863 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4b1192-4aed-4a55-80fd-f199fb102cd3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:07.864340 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4b1192-4aed-4a55-80fd-f199fb102cd3 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:07.953429 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:48:07.977520 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 574/1164] 10.222.0.64 () {58 vars in 1282 bytes} [Tue Aug 2 09:48:07 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 02 09:48:07.984117 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63efeec8-837c-44da-bfdf-fbeb9e5987f7 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:08.032766 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:48:08.088633 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63efeec8-837c-44da-bfdf-fbeb9e5987f7 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:08.125548 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5257633-7ef2-4801-994d-e81350befa52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:08.126188 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5257633-7ef2-4801-994d-e81350befa52 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:08.126468 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5257633-7ef2-4801-994d-e81350befa52 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:08.126741 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5257633-7ef2-4801-994d-e81350befa52 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:08.133645 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5257633-7ef2-4801-994d-e81350befa52 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:08.135096 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5257633-7ef2-4801-994d-e81350befa52 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:08.148308 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-d5257633-7ef2-4801-994d-e81350befa52 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 02 09:48:08.148308 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:48:08.148308 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 02 09:48:08.148308 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 02 09:48:08.148308 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 09:48:08.148308 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 09:48:08.148308 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 09:48:08.148308 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 09:48:08.148308 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 02 09:48:08.148308 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:48:08.148308 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 09:48:08.148308 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:48:08.148308 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:48:08.148308 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:48:08.148308 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:48:08.148308 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:48:08.148308 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:48:08.150495 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:48:08.150495 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:48:08.150495 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:48:08.150495 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:48:08.150495 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:48:08.150495 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:48:08.150495 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 09:48:08.150495 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 09:48:08.150495 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 09:48:08.150495 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 09:48:08.150495 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:48:08.150495 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:48:08.150495 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:48:08.150495 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:48:08.150495 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:48:08.150495 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:48:08.152062 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:48:08.152062 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:48:08.152062 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:48:08.152062 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:48:08.152062 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:48:08.152062 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:48:08.152062 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:48:08.152062 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:48:08.152062 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:48:08.152062 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:48:08.152062 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 02 09:48:08.152062 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 09:48:08.152062 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 02 09:48:08.152062 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:48:08.152062 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 591/1165] 10.222.0.64 () {56 vars in 1117 bytes} [Tue Aug 2 09:48:07 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:48:08.155666 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4b1192-4aed-4a55-80fd-f199fb102cd3 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:08.256544 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4b1192-4aed-4a55-80fd-f199fb102cd3 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:08.291469 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d43a92-7070-4975-967a-f12fda47aa90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:08.292071 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d43a92-7070-4975-967a-f12fda47aa90 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:08.292302 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d43a92-7070-4975-967a-f12fda47aa90 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:08.292576 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d43a92-7070-4975-967a-f12fda47aa90 None admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:08.293270 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d43a92-7070-4975-967a-f12fda47aa90 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:48:08.294089 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d43a92-7070-4975-967a-f12fda47aa90 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:08.295458 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d43a92-7070-4975-967a-f12fda47aa90 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:08.303360 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 575/1166] 10.222.0.64 () {56 vars in 1115 bytes} [Tue Aug 2 09:48:08 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:48:10.304070 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ec80cf-7851-4ad6-8a04-a5f0d920b1c7 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:10.304705 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ec80cf-7851-4ad6-8a04-a5f0d920b1c7 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:10.304705 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ec80cf-7851-4ad6-8a04-a5f0d920b1c7 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:10.306064 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 592/1167] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:48:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:48:10.312686 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae56903-7dbb-48a8-8fd0-ef37f6ed421e None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:10.313344 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae56903-7dbb-48a8-8fd0-ef37f6ed421e None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:10.313344 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae56903-7dbb-48a8-8fd0-ef37f6ed421e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:10.332491 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-4ae56903-7dbb-48a8-8fd0-ef37f6ed421e None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:48:10.333709 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-4ae56903-7dbb-48a8-8fd0-ef37f6ed421e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:48:10.373599 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:48:10.408352 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 576/1168] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:48:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 09:48:10.431007 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ec80cf-7851-4ad6-8a04-a5f0d920b1c7 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:10.533475 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ec80cf-7851-4ad6-8a04-a5f0d920b1c7 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:10.535792 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:48:10.570446 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76be7c0-04f7-49ac-b5cb-83f88231dc44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:10.571072 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76be7c0-04f7-49ac-b5cb-83f88231dc44 None admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:10.571357 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76be7c0-04f7-49ac-b5cb-83f88231dc44 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:10.571631 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76be7c0-04f7-49ac-b5cb-83f88231dc44 None admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:10.572398 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76be7c0-04f7-49ac-b5cb-83f88231dc44 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:10.573791 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76be7c0-04f7-49ac-b5cb-83f88231dc44 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:10.591451 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 593/1169] 10.222.0.64 () {60 vars in 1138 bytes} [Tue Aug 2 09:48:10 2022] POST /identity/v3/domains => generated 267 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 09:48:12.654286 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d15e8d-700d-47d9-963c-dbf95fac60d3 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:12.654849 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d15e8d-700d-47d9-963c-dbf95fac60d3 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:12.654849 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d15e8d-700d-47d9-963c-dbf95fac60d3 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:12.656199 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 577/1170] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:48:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:48:12.661321 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea26ca5-a994-48d9-b3eb-d06025b2762c None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:12.661762 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea26ca5-a994-48d9-b3eb-d06025b2762c None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:12.661857 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea26ca5-a994-48d9-b3eb-d06025b2762c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:12.679680 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-8ea26ca5-a994-48d9-b3eb-d06025b2762c None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:48:12.680902 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-8ea26ca5-a994-48d9-b3eb-d06025b2762c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:48:12.749175 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 594/1171] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:48:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 09:48:12.771990 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d15e8d-700d-47d9-963c-dbf95fac60d3 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:12.877334 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d15e8d-700d-47d9-963c-dbf95fac60d3 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:12.913000 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b728d64f-6e5a-45de-b030-9314af85e195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:12.913571 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b728d64f-6e5a-45de-b030-9314af85e195 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:12.913916 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b728d64f-6e5a-45de-b030-9314af85e195 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:12.914130 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b728d64f-6e5a-45de-b030-9314af85e195 None admin] PATH_INFO: `/v3/domains/1e390deaa6164599a6e160e3af852020` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:12.925057 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b728d64f-6e5a-45de-b030-9314af85e195 None admin] RBAC: Authorizing `identity:get_domain(domain_id=1e390deaa6164599a6e160e3af852020)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:12.926495 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b728d64f-6e5a-45de-b030-9314af85e195 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:12.928116 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 578/1172] 10.222.0.64 () {56 vars in 1183 bytes} [Tue Aug 2 09:48:12 2022] GET /identity/v3/domains/1e390deaa6164599a6e160e3af852020 => generated 267 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:48:12.934435 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea26ca5-a994-48d9-b3eb-d06025b2762c None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:13.031678 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:48:13.039286 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea26ca5-a994-48d9-b3eb-d06025b2762c None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:13.076651 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c624b230-f93c-4bec-9e0c-edb67a3dc637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:13.077268 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c624b230-f93c-4bec-9e0c-edb67a3dc637 None admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:13.077555 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c624b230-f93c-4bec-9e0c-edb67a3dc637 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:13.077827 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c624b230-f93c-4bec-9e0c-edb67a3dc637 None admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:13.078507 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c624b230-f93c-4bec-9e0c-edb67a3dc637 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:13.079938 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c624b230-f93c-4bec-9e0c-edb67a3dc637 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:13.108095 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 595/1173] 10.222.0.64 () {60 vars in 1141 bytes} [Tue Aug 2 09:48:12 2022] POST /identity/v3/projects => generated 361 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:48:15.187096 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c581c5f-677a-4432-8976-a10444d53ef4 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:15.187627 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c581c5f-677a-4432-8976-a10444d53ef4 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:15.187627 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c581c5f-677a-4432-8976-a10444d53ef4 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:15.188971 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 579/1174] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:48:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:48:15.194593 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a582ab-5783-4f3a-aa0b-5081c9e8fce7 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:15.195034 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a582ab-5783-4f3a-aa0b-5081c9e8fce7 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:15.195153 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a582ab-5783-4f3a-aa0b-5081c9e8fce7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:15.213229 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-90a582ab-5783-4f3a-aa0b-5081c9e8fce7 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:48:15.214432 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-90a582ab-5783-4f3a-aa0b-5081c9e8fce7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:48:15.292677 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 596/1175] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:48:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 09:48:15.317839 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c581c5f-677a-4432-8976-a10444d53ef4 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:15.356863 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:48:15.433106 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c581c5f-677a-4432-8976-a10444d53ef4 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:15.467223 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142f8ba8-3f76-4c6a-bcd6-690e288a76b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:15.467775 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142f8ba8-3f76-4c6a-bcd6-690e288a76b8 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:15.468064 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142f8ba8-3f76-4c6a-bcd6-690e288a76b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:15.468370 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142f8ba8-3f76-4c6a-bcd6-690e288a76b8 None admin] PATH_INFO: `/v3/domains/1e390deaa6164599a6e160e3af852020` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:15.469620 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142f8ba8-3f76-4c6a-bcd6-690e288a76b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=1e390deaa6164599a6e160e3af852020)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:15.471068 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142f8ba8-3f76-4c6a-bcd6-690e288a76b8 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:15.472980 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 580/1176] 10.222.0.64 () {56 vars in 1183 bytes} [Tue Aug 2 09:48:15 2022] GET /identity/v3/domains/1e390deaa6164599a6e160e3af852020 => generated 267 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:48:15.480118 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a582ab-5783-4f3a-aa0b-5081c9e8fce7 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:15.562835 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:48:15.586355 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a582ab-5783-4f3a-aa0b-5081c9e8fce7 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:15.625864 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebebb7ae-770c-4231-b273-376e017cdd00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:15.626508 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebebb7ae-770c-4231-b273-376e017cdd00 None admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:15.626790 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebebb7ae-770c-4231-b273-376e017cdd00 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:15.627061 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebebb7ae-770c-4231-b273-376e017cdd00 None admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:15.627753 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebebb7ae-770c-4231-b273-376e017cdd00 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:15.629213 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebebb7ae-770c-4231-b273-376e017cdd00 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:15.651853 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 597/1177] 10.222.0.64 () {60 vars in 1132 bytes} [Tue Aug 2 09:48:15 2022] POST /identity/v3/users => generated 311 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:48:17.745371 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ce8649-5d40-4ec0-a227-577dcbbbf3fc None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:17.745944 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ce8649-5d40-4ec0-a227-577dcbbbf3fc None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:17.745944 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ce8649-5d40-4ec0-a227-577dcbbbf3fc None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:17.747280 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 581/1178] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:48:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:48:17.753101 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548cb33c-2841-4ba6-9303-a847cc9f010c None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:17.753882 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548cb33c-2841-4ba6-9303-a847cc9f010c None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:17.753882 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548cb33c-2841-4ba6-9303-a847cc9f010c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:17.771989 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-548cb33c-2841-4ba6-9303-a847cc9f010c None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:48:17.773233 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-548cb33c-2841-4ba6-9303-a847cc9f010c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:48:17.853081 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 598/1179] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:48:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 09:48:17.876742 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ce8649-5d40-4ec0-a227-577dcbbbf3fc None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:17.977290 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:48:17.985277 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ce8649-5d40-4ec0-a227-577dcbbbf3fc None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:18.020047 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbc6a26-ab84-467b-975b-a7df26ab1925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:18.020637 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbc6a26-ab84-467b-975b-a7df26ab1925 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:18.020921 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbc6a26-ab84-467b-975b-a7df26ab1925 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:18.021197 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbc6a26-ab84-467b-975b-a7df26ab1925 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:18.033771 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbc6a26-ab84-467b-975b-a7df26ab1925 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 09:48:18.038705 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-1dbc6a26-ab84-467b-975b-a7df26ab1925 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 09:48:18.038705 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:48:18.038705 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:48:18.038705 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:48:18.038705 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:48:18.038705 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:48:18.038705 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:48:18.038705 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:48:18.038705 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:48:18.038705 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:48:18.038705 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:48:18.038705 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:48:18.038705 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 09:48:18.038705 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 09:48:18.038705 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 09:48:18.038705 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 09:48:18.039931 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 09:48:18.039931 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 09:48:18.039931 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 09:48:18.039931 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 09:48:18.039931 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:48:18.039931 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:48:18.039931 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:48:18.039931 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:48:18.039931 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:48:18.039931 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:48:18.039931 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:48:18.039931 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:48:18.039931 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:48:18.039931 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:48:18.039931 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:48:18.039931 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:48:18.041258 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:48:18.041258 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:48:18.041258 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:48:18.041258 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:48:18.041258 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 09:48:18.041258 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 09:48:18.041258 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 09:48:18.041258 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 09:48:18.041258 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 09:48:18.041258 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 09:48:18.041258 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 09:48:18.041258 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:48:18.041258 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 582/1180] 10.222.0.64 () {56 vars in 1096 bytes} [Tue Aug 2 09:48:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:48:18.046124 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548cb33c-2841-4ba6-9303-a847cc9f010c None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:18.107278 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:48:18.147958 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548cb33c-2841-4ba6-9303-a847cc9f010c None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:18.182979 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5f782a-187a-4abc-8800-a56ad96cb304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:18.183607 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5f782a-187a-4abc-8800-a56ad96cb304 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:18.184486 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5f782a-187a-4abc-8800-a56ad96cb304 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:18.184769 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5f782a-187a-4abc-8800-a56ad96cb304 None admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:18.185504 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5f782a-187a-4abc-8800-a56ad96cb304 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:48:18.185814 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5f782a-187a-4abc-8800-a56ad96cb304 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:18.187209 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5f782a-187a-4abc-8800-a56ad96cb304 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:18.195865 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 599/1181] 10.222.0.64 () {56 vars in 1099 bytes} [Tue Aug 2 09:48:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:48:18.202106 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbc6a26-ab84-467b-975b-a7df26ab1925 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:18.303877 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbc6a26-ab84-467b-975b-a7df26ab1925 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:18.344175 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f9c1ce-417f-4a28-9880-270d851caf45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:18.344858 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f9c1ce-417f-4a28-9880-270d851caf45 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:18.345144 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f9c1ce-417f-4a28-9880-270d851caf45 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:18.345421 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f9c1ce-417f-4a28-9880-270d851caf45 None admin] PATH_INFO: `/v3/users/e07d14d0b8c446f580163dba05dedd5e` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:18.360238 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f9c1ce-417f-4a28-9880-270d851caf45 None admin] RBAC: Authorizing `identity:get_user(user_id=e07d14d0b8c446f580163dba05dedd5e)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:18.361930 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f9c1ce-417f-4a28-9880-270d851caf45 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:18.364004 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 583/1182] 10.222.0.64 () {56 vars in 1177 bytes} [Tue Aug 2 09:48:18 2022] GET /identity/v3/users/e07d14d0b8c446f580163dba05dedd5e => generated 311 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:48:18.370106 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5f782a-187a-4abc-8800-a56ad96cb304 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:18.479080 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5f782a-187a-4abc-8800-a56ad96cb304 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:18.503377 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:48:18.514800 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9089ea7d-0b90-4f7c-b8fa-07343b492cfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:18.515422 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9089ea7d-0b90-4f7c-b8fa-07343b492cfa None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:18.515707 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9089ea7d-0b90-4f7c-b8fa-07343b492cfa None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:18.515985 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9089ea7d-0b90-4f7c-b8fa-07343b492cfa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:18.584607 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9089ea7d-0b90-4f7c-b8fa-07343b492cfa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:18.586086 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9089ea7d-0b90-4f7c-b8fa-07343b492cfa None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:18.692751 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 600/1183] 10.222.0.64 () {58 vars in 1282 bytes} [Tue Aug 2 09:48:18 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 02 09:48:18.698655 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f9c1ce-417f-4a28-9880-270d851caf45 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:18.717079 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:48:18.800553 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f9c1ce-417f-4a28-9880-270d851caf45 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:18.836615 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7b50fd-6c4c-406f-86b7-c38ff290144a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:18.837165 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7b50fd-6c4c-406f-86b7-c38ff290144a None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:18.837447 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7b50fd-6c4c-406f-86b7-c38ff290144a None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:18.837714 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7b50fd-6c4c-406f-86b7-c38ff290144a None admin] PATH_INFO: `/v3/projects/13852ae77798421bb74f91fee143afb0` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:18.838883 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7b50fd-6c4c-406f-86b7-c38ff290144a None admin] RBAC: Authorizing `identity:get_project(project_id=13852ae77798421bb74f91fee143afb0)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:18.840913 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7b50fd-6c4c-406f-86b7-c38ff290144a None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:18.842657 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 584/1184] 10.222.0.64 () {56 vars in 1186 bytes} [Tue Aug 2 09:48:18 2022] GET /identity/v3/projects/13852ae77798421bb74f91fee143afb0 => generated 361 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:48:18.849187 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9089ea7d-0b90-4f7c-b8fa-07343b492cfa None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:18.874119 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:48:18.951391 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9089ea7d-0b90-4f7c-b8fa-07343b492cfa None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:18.986471 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cc0650-9480-4846-b84c-592245c18a7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:18.987088 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cc0650-9480-4846-b84c-592245c18a7d None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:18.987371 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cc0650-9480-4846-b84c-592245c18a7d None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:18.987645 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cc0650-9480-4846-b84c-592245c18a7d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:18.988466 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cc0650-9480-4846-b84c-592245c18a7d None admin] RBAC: Adding query filter params (role.id=1000cf318fa14915b8091599eaead5de, scope.project.id=13852ae77798421bb74f91fee143afb0, user.id=e07d14d0b8c446f580163dba05dedd5e) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:48:18.988778 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cc0650-9480-4846-b84c-592245c18a7d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:18.990369 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cc0650-9480-4846-b84c-592245c18a7d None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:18.996432 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 601/1185] 10.222.0.64 () {56 vars in 1374 bytes} [Tue Aug 2 09:48:18 2022] GET /identity/v3/role_assignments?user.id=e07d14d0b8c446f580163dba05dedd5e&scope.project.id=13852ae77798421bb74f91fee143afb0&role.id=1000cf318fa14915b8091599eaead5de => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:48:20.995038 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcef8675-1e02-4976-b9e1-8f2fb7ca4487 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:20.995574 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcef8675-1e02-4976-b9e1-8f2fb7ca4487 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:20.995574 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcef8675-1e02-4976-b9e1-8f2fb7ca4487 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:20.997623 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 585/1186] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:48:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:48:21.003316 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b99ff1-aad0-4347-911a-ca12cf674a6b None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:21.003932 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b99ff1-aad0-4347-911a-ca12cf674a6b None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:21.003932 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b99ff1-aad0-4347-911a-ca12cf674a6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:21.022915 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-b3b99ff1-aad0-4347-911a-ca12cf674a6b None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:48:21.024112 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-b3b99ff1-aad0-4347-911a-ca12cf674a6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:48:21.093082 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 602/1187] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:48:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 09:48:21.115087 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcef8675-1e02-4976-b9e1-8f2fb7ca4487 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:21.203734 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:48:21.221869 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcef8675-1e02-4976-b9e1-8f2fb7ca4487 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:21.255607 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26af6011-9c48-4620-8fa8-3ef0e114dbd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:21.256271 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26af6011-9c48-4620-8fa8-3ef0e114dbd8 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:21.256552 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26af6011-9c48-4620-8fa8-3ef0e114dbd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:21.256821 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26af6011-9c48-4620-8fa8-3ef0e114dbd8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:21.267384 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26af6011-9c48-4620-8fa8-3ef0e114dbd8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 09:48:21.272195 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-26af6011-9c48-4620-8fa8-3ef0e114dbd8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 09:48:21.272195 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:48:21.272195 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:48:21.272195 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:48:21.272195 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:48:21.272195 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:48:21.272195 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:48:21.272195 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:48:21.272195 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:48:21.272195 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:48:21.272195 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:48:21.272195 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:48:21.272195 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 09:48:21.272195 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 09:48:21.272195 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 09:48:21.272195 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 09:48:21.273311 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 09:48:21.273311 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 09:48:21.273311 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 09:48:21.273311 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 09:48:21.273311 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:48:21.273311 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:48:21.273311 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:48:21.273311 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:48:21.273311 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:48:21.273311 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:48:21.273311 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:48:21.273311 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:48:21.273311 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:48:21.273311 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:48:21.273311 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:48:21.273311 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:48:21.274374 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:48:21.274374 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:48:21.274374 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:48:21.274374 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:48:21.274374 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 09:48:21.274374 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 09:48:21.274374 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 09:48:21.274374 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 09:48:21.274374 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 09:48:21.274374 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 09:48:21.274374 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 09:48:21.274374 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:48:21.274374 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 586/1188] 10.222.0.64 () {56 vars in 1096 bytes} [Tue Aug 2 09:48:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:48:21.284386 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b99ff1-aad0-4347-911a-ca12cf674a6b None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:21.302898 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:48:21.385632 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b99ff1-aad0-4347-911a-ca12cf674a6b None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:21.420707 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f719b488-d50d-49d8-805b-b7850e385e8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:21.421324 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f719b488-d50d-49d8-805b-b7850e385e8d None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:21.421605 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f719b488-d50d-49d8-805b-b7850e385e8d None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:21.421877 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f719b488-d50d-49d8-805b-b7850e385e8d None admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:21.422588 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f719b488-d50d-49d8-805b-b7850e385e8d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:48:21.422888 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f719b488-d50d-49d8-805b-b7850e385e8d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:21.424279 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f719b488-d50d-49d8-805b-b7850e385e8d None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:21.432602 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 603/1189] 10.222.0.64 () {56 vars in 1099 bytes} [Tue Aug 2 09:48:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:48:21.439245 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26af6011-9c48-4620-8fa8-3ef0e114dbd8 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:21.546901 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26af6011-9c48-4620-8fa8-3ef0e114dbd8 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:21.581244 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c96f6d-d508-4139-904c-eebac065c8d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:21.581886 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c96f6d-d508-4139-904c-eebac065c8d6 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:21.582168 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c96f6d-d508-4139-904c-eebac065c8d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:21.582441 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c96f6d-d508-4139-904c-eebac065c8d6 None admin] PATH_INFO: `/v3/users/e07d14d0b8c446f580163dba05dedd5e` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:21.583633 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c96f6d-d508-4139-904c-eebac065c8d6 None admin] RBAC: Authorizing `identity:get_user(user_id=e07d14d0b8c446f580163dba05dedd5e)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:21.585092 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c96f6d-d508-4139-904c-eebac065c8d6 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:21.586787 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 587/1190] 10.222.0.64 () {56 vars in 1177 bytes} [Tue Aug 2 09:48:21 2022] GET /identity/v3/users/e07d14d0b8c446f580163dba05dedd5e => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:48:21.592943 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f719b488-d50d-49d8-805b-b7850e385e8d None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:21.675176 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:48:21.695463 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f719b488-d50d-49d8-805b-b7850e385e8d None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:21.731060 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a17c5ad-a3dd-4bbc-8bcb-55bef251e39e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:21.731708 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a17c5ad-a3dd-4bbc-8bcb-55bef251e39e None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:21.731995 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a17c5ad-a3dd-4bbc-8bcb-55bef251e39e None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:21.732350 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a17c5ad-a3dd-4bbc-8bcb-55bef251e39e None admin] PATH_INFO: `/v3/projects/13852ae77798421bb74f91fee143afb0` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:21.733490 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a17c5ad-a3dd-4bbc-8bcb-55bef251e39e None admin] RBAC: Authorizing `identity:get_project(project_id=13852ae77798421bb74f91fee143afb0)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:21.734986 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a17c5ad-a3dd-4bbc-8bcb-55bef251e39e None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:21.736740 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 604/1191] 10.222.0.64 () {56 vars in 1186 bytes} [Tue Aug 2 09:48:21 2022] GET /identity/v3/projects/13852ae77798421bb74f91fee143afb0 => generated 361 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:48:21.743007 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c96f6d-d508-4139-904c-eebac065c8d6 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:21.758818 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:48:21.849492 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c96f6d-d508-4139-904c-eebac065c8d6 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:21.888086 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69df3a34-a205-4b84-b119-14614c62cdca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:21.888628 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69df3a34-a205-4b84-b119-14614c62cdca None admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:21.888918 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69df3a34-a205-4b84-b119-14614c62cdca None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:21.889196 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69df3a34-a205-4b84-b119-14614c62cdca None admin] PATH_INFO: `/v3/projects/13852ae77798421bb74f91fee143afb0/users/e07d14d0b8c446f580163dba05dedd5e/roles/1000cf318fa14915b8091599eaead5de` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:21.891193 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69df3a34-a205-4b84-b119-14614c62cdca None admin] RBAC: Authorizing `identity:create_grant(project_id=13852ae77798421bb74f91fee143afb0, user_id=e07d14d0b8c446f580163dba05dedd5e, role_id=1000cf318fa14915b8091599eaead5de)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:21.892679 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69df3a34-a205-4b84-b119-14614c62cdca None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:21.906689 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 588/1192] 10.222.0.64 () {58 vars in 1439 bytes} [Tue Aug 2 09:48:21 2022] PUT /identity/v3/projects/13852ae77798421bb74f91fee143afb0/users/e07d14d0b8c446f580163dba05dedd5e/roles/1000cf318fa14915b8091599eaead5de => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:48:23.932800 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8083b13c-83b9-4420-9a89-54ad062ae097 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:23.933366 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8083b13c-83b9-4420-9a89-54ad062ae097 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:23.933366 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8083b13c-83b9-4420-9a89-54ad062ae097 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:23.935236 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 605/1193] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:48:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:48:23.940811 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d41f8a-509a-46c1-bd24-e46e65b08d30 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:23.941312 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d41f8a-509a-46c1-bd24-e46e65b08d30 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:23.941413 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d41f8a-509a-46c1-bd24-e46e65b08d30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:23.960079 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-71d41f8a-509a-46c1-bd24-e46e65b08d30 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:48:23.961317 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-71d41f8a-509a-46c1-bd24-e46e65b08d30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:48:24.030577 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 589/1194] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:48:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 09:48:24.054073 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8083b13c-83b9-4420-9a89-54ad062ae097 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:24.160327 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8083b13c-83b9-4420-9a89-54ad062ae097 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:24.194509 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5dd758-94b1-43e0-afa8-e9dd61e99b4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:24.195145 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5dd758-94b1-43e0-afa8-e9dd61e99b4b None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:24.195431 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5dd758-94b1-43e0-afa8-e9dd61e99b4b None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:24.195709 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5dd758-94b1-43e0-afa8-e9dd61e99b4b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:24.206390 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5dd758-94b1-43e0-afa8-e9dd61e99b4b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 09:48:24.211252 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-8a5dd758-94b1-43e0-afa8-e9dd61e99b4b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 09:48:24.211252 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:48:24.211252 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:48:24.211252 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:48:24.211252 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:48:24.211252 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:48:24.211252 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:48:24.211252 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:48:24.211252 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:48:24.211252 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:48:24.211252 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:48:24.211252 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:48:24.211252 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 09:48:24.211252 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 09:48:24.211252 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 09:48:24.211252 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 09:48:24.212418 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 09:48:24.212418 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 09:48:24.212418 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 09:48:24.212418 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 09:48:24.212418 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:48:24.212418 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:48:24.212418 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:48:24.212418 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:48:24.212418 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:48:24.212418 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:48:24.212418 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:48:24.212418 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:48:24.212418 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:48:24.212418 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:48:24.212418 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:48:24.212418 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:48:24.213434 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:48:24.213434 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:48:24.213434 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:48:24.213434 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:48:24.213434 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 09:48:24.213434 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 09:48:24.213434 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 09:48:24.213434 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 09:48:24.213434 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 09:48:24.213434 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 09:48:24.213434 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 09:48:24.213434 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:48:24.213434 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 606/1195] 10.222.0.64 () {56 vars in 1096 bytes} [Tue Aug 2 09:48:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:48:24.218624 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d41f8a-509a-46c1-bd24-e46e65b08d30 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:24.220850 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:48:24.322023 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d41f8a-509a-46c1-bd24-e46e65b08d30 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:24.356749 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b5ff98-e1bc-46f8-80af-4de62a7079e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:24.357375 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b5ff98-e1bc-46f8-80af-4de62a7079e8 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:24.357664 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b5ff98-e1bc-46f8-80af-4de62a7079e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:24.357944 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b5ff98-e1bc-46f8-80af-4de62a7079e8 None admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:24.358675 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b5ff98-e1bc-46f8-80af-4de62a7079e8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:48:24.358985 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b5ff98-e1bc-46f8-80af-4de62a7079e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:24.360443 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b5ff98-e1bc-46f8-80af-4de62a7079e8 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:24.369158 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 590/1196] 10.222.0.64 () {56 vars in 1099 bytes} [Tue Aug 2 09:48:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:48:24.375624 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5dd758-94b1-43e0-afa8-e9dd61e99b4b None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:24.387377 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:48:24.476327 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5dd758-94b1-43e0-afa8-e9dd61e99b4b None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:24.520831 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e947fe9-1cf1-4a08-a959-9708efcddb86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:24.521585 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e947fe9-1cf1-4a08-a959-9708efcddb86 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:24.521873 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e947fe9-1cf1-4a08-a959-9708efcddb86 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:24.522149 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e947fe9-1cf1-4a08-a959-9708efcddb86 None admin] PATH_INFO: `/v3/users/e07d14d0b8c446f580163dba05dedd5e` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:24.523322 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e947fe9-1cf1-4a08-a959-9708efcddb86 None admin] RBAC: Authorizing `identity:get_user(user_id=e07d14d0b8c446f580163dba05dedd5e)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:24.527180 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e947fe9-1cf1-4a08-a959-9708efcddb86 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:24.527180 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 607/1197] 10.222.0.64 () {56 vars in 1177 bytes} [Tue Aug 2 09:48:24 2022] GET /identity/v3/users/e07d14d0b8c446f580163dba05dedd5e => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:48:24.532399 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b5ff98-e1bc-46f8-80af-4de62a7079e8 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:24.597482 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:48:24.636534 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b5ff98-e1bc-46f8-80af-4de62a7079e8 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:24.674715 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b224d9f1-9229-4b23-864d-d0a5a6a0c860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:24.675341 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b224d9f1-9229-4b23-864d-d0a5a6a0c860 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:24.675631 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b224d9f1-9229-4b23-864d-d0a5a6a0c860 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:24.675905 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b224d9f1-9229-4b23-864d-d0a5a6a0c860 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:24.753205 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b224d9f1-9229-4b23-864d-d0a5a6a0c860 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:24.754592 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b224d9f1-9229-4b23-864d-d0a5a6a0c860 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:24.817884 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:48:24.858351 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 591/1198] 10.222.0.64 () {58 vars in 1282 bytes} [Tue Aug 2 09:48:24 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 02 09:48:24.864332 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e947fe9-1cf1-4a08-a959-9708efcddb86 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:24.943345 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:48:24.968778 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e947fe9-1cf1-4a08-a959-9708efcddb86 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:25.003527 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f232beb-096a-4ade-85ed-f66e130244ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:25.004078 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f232beb-096a-4ade-85ed-f66e130244ab None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:25.004393 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f232beb-096a-4ade-85ed-f66e130244ab None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:25.004670 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f232beb-096a-4ade-85ed-f66e130244ab None admin] PATH_INFO: `/v3/projects/13852ae77798421bb74f91fee143afb0` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:25.005940 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f232beb-096a-4ade-85ed-f66e130244ab None admin] RBAC: Authorizing `identity:get_project(project_id=13852ae77798421bb74f91fee143afb0)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:25.007398 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f232beb-096a-4ade-85ed-f66e130244ab None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:25.009107 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 608/1199] 10.222.0.64 () {56 vars in 1186 bytes} [Tue Aug 2 09:48:24 2022] GET /identity/v3/projects/13852ae77798421bb74f91fee143afb0 => generated 361 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:48:25.018495 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b224d9f1-9229-4b23-864d-d0a5a6a0c860 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:48:25.122969 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b224d9f1-9229-4b23-864d-d0a5a6a0c860 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:48:25.169222 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a9a2c4-cab4-457f-8aba-2df9e4c88d5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:48:25.169845 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a9a2c4-cab4-457f-8aba-2df9e4c88d5e None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:48:25.170129 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a9a2c4-cab4-457f-8aba-2df9e4c88d5e None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:48:25.170405 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a9a2c4-cab4-457f-8aba-2df9e4c88d5e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:48:25.171209 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a9a2c4-cab4-457f-8aba-2df9e4c88d5e None admin] RBAC: Adding query filter params (role.id=1000cf318fa14915b8091599eaead5de, scope.project.id=13852ae77798421bb74f91fee143afb0, user.id=e07d14d0b8c446f580163dba05dedd5e) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:48:25.171525 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a9a2c4-cab4-457f-8aba-2df9e4c88d5e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:48:25.172995 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a9a2c4-cab4-457f-8aba-2df9e4c88d5e None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:48:25.179290 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 592/1200] 10.222.0.64 () {56 vars in 1374 bytes} [Tue Aug 2 09:48:25 2022] GET /identity/v3/role_assignments?user.id=e07d14d0b8c446f580163dba05dedd5e&scope.project.id=13852ae77798421bb74f91fee143afb0&role.id=1000cf318fa14915b8091599eaead5de => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:48:34.999230 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=28 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:48:35.176508 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=86 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:49:21.381225 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c355b1-b202-4747-9992-c8ca7e7828ce None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:21.381225 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c355b1-b202-4747-9992-c8ca7e7828ce None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:21.381225 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c355b1-b202-4747-9992-c8ca7e7828ce None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:21.381225 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 609/1201] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:49:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:49:21.387288 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc4ff6a-028b-45c3-80b7-d94c4550ed3b None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:21.387783 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc4ff6a-028b-45c3-80b7-d94c4550ed3b None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:21.387891 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc4ff6a-028b-45c3-80b7-d94c4550ed3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:21.410324 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-7fc4ff6a-028b-45c3-80b7-d94c4550ed3b None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:49:21.411633 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-7fc4ff6a-028b-45c3-80b7-d94c4550ed3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:49:21.513056 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 593/1202] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:49:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 09:49:21.539566 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c355b1-b202-4747-9992-c8ca7e7828ce None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:49:21.676937 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c355b1-b202-4747-9992-c8ca7e7828ce None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:49:21.715180 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f52e79-ecfc-413b-b87b-dfcb5caffc66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:49:21.715921 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f52e79-ecfc-413b-b87b-dfcb5caffc66 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:21.716204 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f52e79-ecfc-413b-b87b-dfcb5caffc66 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:21.716509 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f52e79-ecfc-413b-b87b-dfcb5caffc66 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:21.729199 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f52e79-ecfc-413b-b87b-dfcb5caffc66 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:49:21.730716 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f52e79-ecfc-413b-b87b-dfcb5caffc66 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:49:21.750812 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-33f52e79-ecfc-413b-b87b-dfcb5caffc66 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:49:21.750812 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:49:21.750812 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:49:21.750812 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:49:21.750812 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:49:21.750812 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:49:21.750812 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:49:21.750812 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:49:21.750812 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:49:21.750812 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:49:21.750812 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:49:21.750812 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:49:21.750812 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 09:49:21.750812 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 09:49:21.750812 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 09:49:21.751885 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 09:49:21.751885 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:49:21.751885 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:49:21.751885 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:49:21.751885 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:49:21.751885 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:49:21.751885 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:49:21.751885 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:49:21.751885 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:49:21.751885 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:49:21.751885 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:49:21.751885 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:49:21.751885 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:49:21.751885 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:49:21.751885 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:49:21.751885 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:49:21.751885 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:49:21.752924 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 09:49:21.752924 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 09:49:21.752924 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:49:21.752924 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:49:21.756410 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 610/1203] 10.222.0.64 () {56 vars in 1108 bytes} [Tue Aug 2 09:49:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:49:21.758923 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc4ff6a-028b-45c3-80b7-d94c4550ed3b None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:49:21.894125 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc4ff6a-028b-45c3-80b7-d94c4550ed3b None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:49:21.942056 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:49:21.957960 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b35297-0bf4-4fac-b322-31ec86330e7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:49:21.958616 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b35297-0bf4-4fac-b322-31ec86330e7d None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:21.958904 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b35297-0bf4-4fac-b322-31ec86330e7d None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:21.959211 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b35297-0bf4-4fac-b322-31ec86330e7d None admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:21.959936 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b35297-0bf4-4fac-b322-31ec86330e7d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:49:21.960267 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b35297-0bf4-4fac-b322-31ec86330e7d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:49:21.961733 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b35297-0bf4-4fac-b322-31ec86330e7d None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:49:21.977902 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 594/1204] 10.222.0.64 () {56 vars in 1109 bytes} [Tue Aug 2 09:49:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:49:21.984157 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f52e79-ecfc-413b-b87b-dfcb5caffc66 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:49:22.085624 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:49:22.158799 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f52e79-ecfc-413b-b87b-dfcb5caffc66 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:49:22.209439 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066c4efb-ff01-4ba5-ba6b-4159335cf26e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:49:22.210033 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066c4efb-ff01-4ba5-ba6b-4159335cf26e None admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:22.210653 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066c4efb-ff01-4ba5-ba6b-4159335cf26e None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:22.211069 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066c4efb-ff01-4ba5-ba6b-4159335cf26e None admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:22.211772 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066c4efb-ff01-4ba5-ba6b-4159335cf26e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:49:22.213635 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066c4efb-ff01-4ba5-ba6b-4159335cf26e None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:49:22.247293 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 611/1205] 10.222.0.64 () {60 vars in 1132 bytes} [Tue Aug 2 09:49:21 2022] POST /identity/v3/users => generated 251 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 09:49:24.510993 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47174896-6fb9-4302-8dcc-75dc490e6895 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:24.511550 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47174896-6fb9-4302-8dcc-75dc490e6895 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:24.511550 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47174896-6fb9-4302-8dcc-75dc490e6895 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:24.512976 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 595/1206] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:49:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:49:24.518269 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925c3338-0007-47b1-8379-8dac0880f99c None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:24.521486 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925c3338-0007-47b1-8379-8dac0880f99c None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:24.521486 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925c3338-0007-47b1-8379-8dac0880f99c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:24.537707 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-925c3338-0007-47b1-8379-8dac0880f99c None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:49:24.538872 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-925c3338-0007-47b1-8379-8dac0880f99c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:49:24.618907 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 612/1207] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:49:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 09:49:24.639871 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47174896-6fb9-4302-8dcc-75dc490e6895 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:49:24.758560 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47174896-6fb9-4302-8dcc-75dc490e6895 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:49:24.797687 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a9680b-44ae-4b34-ab13-9e013fa17bb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:49:24.798286 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a9680b-44ae-4b34-ab13-9e013fa17bb8 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:24.798572 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a9680b-44ae-4b34-ab13-9e013fa17bb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:24.798847 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a9680b-44ae-4b34-ab13-9e013fa17bb8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:24.812283 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a9680b-44ae-4b34-ab13-9e013fa17bb8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 09:49:24.818461 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-73a9680b-44ae-4b34-ab13-9e013fa17bb8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 09:49:24.818461 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:49:24.818461 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:49:24.818461 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:49:24.818461 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:49:24.818461 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:49:24.818461 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:49:24.818461 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:49:24.818461 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:49:24.818461 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:49:24.818461 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:49:24.818461 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:49:24.818461 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 09:49:24.818461 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 09:49:24.818461 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 09:49:24.818461 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 09:49:24.819550 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 09:49:24.819550 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 09:49:24.819550 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 09:49:24.819550 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 09:49:24.819550 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:49:24.819550 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:49:24.819550 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:49:24.819550 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:49:24.819550 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:49:24.819550 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:49:24.819550 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:49:24.819550 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:49:24.819550 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:49:24.819550 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:49:24.819550 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:49:24.819550 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:49:24.820829 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:49:24.820829 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:49:24.820829 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:49:24.820829 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:49:24.820829 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 09:49:24.820829 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 09:49:24.820829 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 09:49:24.820829 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 09:49:24.820829 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 09:49:24.820829 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 09:49:24.820829 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 09:49:24.820829 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:49:24.820829 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 596/1208] 10.222.0.64 () {56 vars in 1102 bytes} [Tue Aug 2 09:49:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 09:49:24.826644 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925c3338-0007-47b1-8379-8dac0880f99c None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:49:24.877916 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:49:24.949429 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925c3338-0007-47b1-8379-8dac0880f99c None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:49:24.987843 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346b85b8-538a-42a7-9b39-acb044177be4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:49:24.988491 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346b85b8-538a-42a7-9b39-acb044177be4 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:24.989395 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346b85b8-538a-42a7-9b39-acb044177be4 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:24.989395 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346b85b8-538a-42a7-9b39-acb044177be4 None admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:24.989736 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346b85b8-538a-42a7-9b39-acb044177be4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:49:24.990033 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346b85b8-538a-42a7-9b39-acb044177be4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:49:24.991398 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346b85b8-538a-42a7-9b39-acb044177be4 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:49:25.000188 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 613/1209] 10.222.0.64 () {56 vars in 1103 bytes} [Tue Aug 2 09:49:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:49:25.006866 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a9680b-44ae-4b34-ab13-9e013fa17bb8 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:49:25.063991 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:49:25.129440 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a9680b-44ae-4b34-ab13-9e013fa17bb8 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:49:25.178610 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423b7328-fed3-4557-8d6c-fbd5a0bdec69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:49:25.179241 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423b7328-fed3-4557-8d6c-fbd5a0bdec69 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:25.179520 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423b7328-fed3-4557-8d6c-fbd5a0bdec69 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:25.179793 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423b7328-fed3-4557-8d6c-fbd5a0bdec69 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:25.191930 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423b7328-fed3-4557-8d6c-fbd5a0bdec69 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:49:25.193565 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423b7328-fed3-4557-8d6c-fbd5a0bdec69 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:49:25.209405 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-423b7328-fed3-4557-8d6c-fbd5a0bdec69 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:49:25.209405 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:49:25.209405 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:49:25.209405 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:49:25.209405 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:49:25.209405 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:49:25.209405 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:49:25.209405 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:49:25.209405 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:49:25.209405 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:49:25.209405 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:49:25.209405 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:49:25.209405 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 09:49:25.209405 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 09:49:25.209405 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 09:49:25.210585 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 09:49:25.210585 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:49:25.210585 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:49:25.210585 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:49:25.210585 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:49:25.210585 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:49:25.210585 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:49:25.210585 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:49:25.210585 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:49:25.210585 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:49:25.210585 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:49:25.210585 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:49:25.210585 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:49:25.210585 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:49:25.210585 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:49:25.210585 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:49:25.210585 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:49:25.213817 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 09:49:25.213817 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 09:49:25.213817 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:49:25.213817 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:49:25.213817 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 597/1210] 10.222.0.64 () {56 vars in 1108 bytes} [Tue Aug 2 09:49:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:49:25.216818 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346b85b8-538a-42a7-9b39-acb044177be4 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:49:25.327933 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:49:25.329638 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346b85b8-538a-42a7-9b39-acb044177be4 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:49:25.368269 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b317d1d-20d3-44d9-ba2d-2e271d750a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:49:25.368947 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b317d1d-20d3-44d9-ba2d-2e271d750a06 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:25.369234 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b317d1d-20d3-44d9-ba2d-2e271d750a06 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:25.369509 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b317d1d-20d3-44d9-ba2d-2e271d750a06 None admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:25.370208 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b317d1d-20d3-44d9-ba2d-2e271d750a06 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:49:25.370502 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b317d1d-20d3-44d9-ba2d-2e271d750a06 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:49:25.371859 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b317d1d-20d3-44d9-ba2d-2e271d750a06 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:49:25.386470 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 614/1211] 10.222.0.64 () {56 vars in 1109 bytes} [Tue Aug 2 09:49:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:49:25.393092 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423b7328-fed3-4557-8d6c-fbd5a0bdec69 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:49:25.507740 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423b7328-fed3-4557-8d6c-fbd5a0bdec69 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:49:25.510114 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:49:25.547015 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0327db-7f1a-44ad-848c-24fbb1220fb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:49:25.547962 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0327db-7f1a-44ad-848c-24fbb1220fb1 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:25.547962 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0327db-7f1a-44ad-848c-24fbb1220fb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:25.548190 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0327db-7f1a-44ad-848c-24fbb1220fb1 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:25.557012 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0327db-7f1a-44ad-848c-24fbb1220fb1 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:49:25.558549 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0327db-7f1a-44ad-848c-24fbb1220fb1 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:49:25.570645 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-4b0327db-7f1a-44ad-848c-24fbb1220fb1 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 02 09:49:25.570645 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:49:25.570645 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:49:25.570645 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:49:25.570645 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:49:25.570645 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:49:25.570645 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:49:25.570645 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:49:25.570645 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:49:25.570645 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:49:25.570645 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:49:25.570645 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:49:25.570645 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 09:49:25.570645 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 09:49:25.570645 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 09:49:25.570645 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 09:49:25.571801 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:49:25.571801 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:49:25.571801 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 09:49:25.571801 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 09:49:25.571801 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 09:49:25.571801 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 09:49:25.571801 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:49:25.571801 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:49:25.571801 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:49:25.571801 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:49:25.571801 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:49:25.571801 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:49:25.571801 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:49:25.571801 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:49:25.571801 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:49:25.571801 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:49:25.571801 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:49:25.571801 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:49:25.573200 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:49:25.573200 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:49:25.573200 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 09:49:25.573200 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 09:49:25.573200 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 09:49:25.573200 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 09:49:25.573200 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 09:49:25.573200 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 09:49:25.573200 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 02 09:49:25.573200 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:49:25.573200 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 598/1212] 10.222.0.64 () {56 vars in 1108 bytes} [Tue Aug 2 09:49:25 2022] GET /identity/v3/users/placement => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:49:25.578612 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b317d1d-20d3-44d9-ba2d-2e271d750a06 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:49:25.711719 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b317d1d-20d3-44d9-ba2d-2e271d750a06 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:49:25.754121 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412981da-ff75-4a40-82db-039381980382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:49:25.755097 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412981da-ff75-4a40-82db-039381980382 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:25.755788 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412981da-ff75-4a40-82db-039381980382 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:25.756140 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412981da-ff75-4a40-82db-039381980382 None admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:25.757197 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412981da-ff75-4a40-82db-039381980382 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:49:25.757568 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412981da-ff75-4a40-82db-039381980382 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:49:25.759204 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412981da-ff75-4a40-82db-039381980382 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:49:25.780273 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 615/1213] 10.222.0.64 () {56 vars in 1143 bytes} [Tue Aug 2 09:49:25 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:49:25.785923 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0327db-7f1a-44ad-848c-24fbb1220fb1 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:49:25.937816 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0327db-7f1a-44ad-848c-24fbb1220fb1 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:49:25.979858 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fd0ff3-1df9-444b-897e-1ef75b5ae473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:49:25.980608 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fd0ff3-1df9-444b-897e-1ef75b5ae473 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:25.980900 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fd0ff3-1df9-444b-897e-1ef75b5ae473 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:25.981180 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fd0ff3-1df9-444b-897e-1ef75b5ae473 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:26.015914 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:49:26.059678 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fd0ff3-1df9-444b-897e-1ef75b5ae473 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:49:26.061309 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fd0ff3-1df9-444b-897e-1ef75b5ae473 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:49:26.182642 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 599/1214] 10.222.0.64 () {58 vars in 1282 bytes} [Tue Aug 2 09:49:25 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 02 09:49:26.188988 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412981da-ff75-4a40-82db-039381980382 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:49:26.211548 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:49:26.311275 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412981da-ff75-4a40-82db-039381980382 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:49:26.353372 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d63702e-3c8d-4b16-9b5e-b6802d5aa074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:49:26.354017 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d63702e-3c8d-4b16-9b5e-b6802d5aa074 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:26.354297 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d63702e-3c8d-4b16-9b5e-b6802d5aa074 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:26.354568 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d63702e-3c8d-4b16-9b5e-b6802d5aa074 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:26.366700 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d63702e-3c8d-4b16-9b5e-b6802d5aa074 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:49:26.368579 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d63702e-3c8d-4b16-9b5e-b6802d5aa074 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:49:26.387144 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-1d63702e-3c8d-4b16-9b5e-b6802d5aa074 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:49:26.387144 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:49:26.387144 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:49:26.387144 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:49:26.387144 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:49:26.387144 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:49:26.387144 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:49:26.387144 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:49:26.387144 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:49:26.387144 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:49:26.387144 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:49:26.387144 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:49:26.387144 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 09:49:26.387144 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 09:49:26.387144 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 09:49:26.389860 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 09:49:26.389860 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:49:26.389860 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:49:26.389860 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:49:26.389860 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:49:26.389860 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:49:26.389860 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:49:26.389860 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:49:26.389860 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:49:26.389860 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:49:26.389860 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:49:26.389860 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:49:26.389860 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:49:26.389860 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:49:26.389860 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:49:26.389860 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:49:26.389860 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:49:26.391188 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 09:49:26.391188 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 09:49:26.391188 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:49:26.391188 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:49:26.391188 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 616/1215] 10.222.0.64 () {56 vars in 1108 bytes} [Tue Aug 2 09:49:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:49:26.395420 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fd0ff3-1df9-444b-897e-1ef75b5ae473 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:49:26.466489 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:49:26.516232 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fd0ff3-1df9-444b-897e-1ef75b5ae473 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:49:26.557742 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fee755-4a44-4b5a-91e8-0c0287e06db3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:49:26.559751 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fee755-4a44-4b5a-91e8-0c0287e06db3 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:26.560755 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fee755-4a44-4b5a-91e8-0c0287e06db3 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:26.561983 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fee755-4a44-4b5a-91e8-0c0287e06db3 None admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:26.562864 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fee755-4a44-4b5a-91e8-0c0287e06db3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:49:26.563565 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fee755-4a44-4b5a-91e8-0c0287e06db3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:49:26.565265 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fee755-4a44-4b5a-91e8-0c0287e06db3 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:49:26.590369 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 600/1216] 10.222.0.64 () {56 vars in 1109 bytes} [Tue Aug 2 09:49:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:49:26.591821 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d63702e-3c8d-4b16-9b5e-b6802d5aa074 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:49:26.683018 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:49:26.724575 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d63702e-3c8d-4b16-9b5e-b6802d5aa074 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:49:26.777885 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfac4d29-fdb1-4676-883e-d08a040b6f2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:49:26.778593 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfac4d29-fdb1-4676-883e-d08a040b6f2a None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:26.778856 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfac4d29-fdb1-4676-883e-d08a040b6f2a None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:26.779163 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfac4d29-fdb1-4676-883e-d08a040b6f2a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:26.801303 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfac4d29-fdb1-4676-883e-d08a040b6f2a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:49:26.802830 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfac4d29-fdb1-4676-883e-d08a040b6f2a None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:49:26.822593 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-cfac4d29-fdb1-4676-883e-d08a040b6f2a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 09:49:26.822593 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:49:26.822593 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:49:26.822593 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:49:26.822593 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:49:26.822593 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:49:26.822593 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:49:26.822593 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:49:26.822593 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:49:26.822593 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:49:26.822593 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:49:26.822593 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:49:26.822593 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 09:49:26.822593 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 09:49:26.822593 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 09:49:26.823626 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 09:49:26.823626 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:49:26.823626 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:49:26.823626 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:49:26.823626 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:49:26.823626 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:49:26.823626 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:49:26.823626 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:49:26.823626 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:49:26.823626 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:49:26.823626 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:49:26.823626 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:49:26.823626 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:49:26.823626 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:49:26.823626 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:49:26.823626 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:49:26.823626 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:49:26.824589 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 09:49:26.824589 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 09:49:26.824589 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 09:49:26.824589 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 09:49:26.824589 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 09:49:26.824589 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 09:49:26.824589 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 09:49:26.824589 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:49:26.830713 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 617/1217] 10.222.0.64 () {56 vars in 1111 bytes} [Tue Aug 2 09:49:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 09:49:26.830879 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fee755-4a44-4b5a-91e8-0c0287e06db3 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:49:26.992478 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fee755-4a44-4b5a-91e8-0c0287e06db3 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:49:27.018537 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:49:27.032201 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f1f1cc-b2fc-478d-b13c-ee277528104d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:49:27.032877 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f1f1cc-b2fc-478d-b13c-ee277528104d None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:27.033162 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f1f1cc-b2fc-478d-b13c-ee277528104d None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:27.033436 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f1f1cc-b2fc-478d-b13c-ee277528104d None admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:27.034196 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f1f1cc-b2fc-478d-b13c-ee277528104d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:49:27.034518 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f1f1cc-b2fc-478d-b13c-ee277528104d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:49:27.035995 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f1f1cc-b2fc-478d-b13c-ee277528104d None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:49:27.055128 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfac4d29-fdb1-4676-883e-d08a040b6f2a None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:49:27.055727 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 601/1218] 10.222.0.64 () {56 vars in 1148 bytes} [Tue Aug 2 09:49:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:49:27.205796 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfac4d29-fdb1-4676-883e-d08a040b6f2a None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:49:27.239148 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:49:27.247761 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da159c6-d3ec-4237-adcc-8ce63cf971d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:49:27.248827 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da159c6-d3ec-4237-adcc-8ce63cf971d3 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:27.249143 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da159c6-d3ec-4237-adcc-8ce63cf971d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:27.249408 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da159c6-d3ec-4237-adcc-8ce63cf971d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:27.250372 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da159c6-d3ec-4237-adcc-8ce63cf971d3 None admin] RBAC: Adding query filter params (role.id=5704af09562a457fa35364e8af9481f1, scope.project.id=390bd7c4f91c4465b0c7ab0f9e436c4d, user.id=b770f073c86b4d88837a8b1f796a8b1d) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:49:27.250668 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da159c6-d3ec-4237-adcc-8ce63cf971d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:49:27.252200 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da159c6-d3ec-4237-adcc-8ce63cf971d3 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:49:27.257932 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 618/1219] 10.222.0.64 () {56 vars in 1374 bytes} [Tue Aug 2 09:49:27 2022] GET /identity/v3/role_assignments?user.id=b770f073c86b4d88837a8b1f796a8b1d&scope.project.id=390bd7c4f91c4465b0c7ab0f9e436c4d&role.id=5704af09562a457fa35364e8af9481f1 => generated 259 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:49:29.700397 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1167f2bf-ec20-49b8-9f86-2bdcd44425af None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:29.701434 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1167f2bf-ec20-49b8-9f86-2bdcd44425af None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:29.701434 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1167f2bf-ec20-49b8-9f86-2bdcd44425af None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:29.702302 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 602/1220] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:49:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 09:49:29.707869 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a48fc8-5d08-4c4d-b6e2-a6e487661654 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:29.708378 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a48fc8-5d08-4c4d-b6e2-a6e487661654 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:29.708480 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a48fc8-5d08-4c4d-b6e2-a6e487661654 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:29.729264 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-d3a48fc8-5d08-4c4d-b6e2-a6e487661654 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:49:29.730458 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-d3a48fc8-5d08-4c4d-b6e2-a6e487661654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:49:29.819333 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 619/1221] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:49:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 09:49:29.842053 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1167f2bf-ec20-49b8-9f86-2bdcd44425af None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:49:29.958511 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1167f2bf-ec20-49b8-9f86-2bdcd44425af None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:49:29.999523 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20682f1c-072f-4395-b123-527a22eb5a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:49:30.000240 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20682f1c-072f-4395-b123-527a22eb5a81 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:30.000678 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20682f1c-072f-4395-b123-527a22eb5a81 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:30.000964 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20682f1c-072f-4395-b123-527a22eb5a81 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:30.012446 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20682f1c-072f-4395-b123-527a22eb5a81 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 09:49:30.017734 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-20682f1c-072f-4395-b123-527a22eb5a81 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 09:49:30.017734 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:49:30.017734 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:49:30.017734 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:49:30.017734 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:49:30.017734 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:49:30.017734 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:49:30.017734 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:49:30.017734 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:49:30.017734 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:49:30.017734 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:49:30.017734 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:49:30.017734 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 09:49:30.017734 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 09:49:30.017734 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 09:49:30.017734 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 09:49:30.019236 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 09:49:30.019236 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 09:49:30.019236 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 09:49:30.019236 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 09:49:30.019236 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:49:30.019236 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:49:30.019236 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:49:30.019236 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:49:30.019236 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:49:30.019236 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:49:30.019236 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:49:30.019236 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:49:30.019236 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:49:30.019236 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:49:30.019236 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:49:30.019236 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:49:30.020341 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:49:30.020341 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:49:30.020341 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:49:30.020341 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:49:30.020341 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 09:49:30.020341 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 09:49:30.020341 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 09:49:30.020341 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 09:49:30.020341 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 09:49:30.020341 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 09:49:30.020341 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 09:49:30.020341 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:49:30.020341 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 603/1222] 10.222.0.64 () {56 vars in 1102 bytes} [Tue Aug 2 09:49:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:49:30.025724 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a48fc8-5d08-4c4d-b6e2-a6e487661654 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:49:30.151934 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a48fc8-5d08-4c4d-b6e2-a6e487661654 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:49:30.178496 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:49:30.190642 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abdfdcc-0dd6-4c40-bafa-936a542777d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:49:30.191261 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abdfdcc-0dd6-4c40-bafa-936a542777d9 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:30.191544 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abdfdcc-0dd6-4c40-bafa-936a542777d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:30.191814 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abdfdcc-0dd6-4c40-bafa-936a542777d9 None admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:30.192566 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abdfdcc-0dd6-4c40-bafa-936a542777d9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:49:30.192867 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abdfdcc-0dd6-4c40-bafa-936a542777d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:49:30.194255 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abdfdcc-0dd6-4c40-bafa-936a542777d9 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:49:30.204821 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 620/1223] 10.222.0.64 () {56 vars in 1103 bytes} [Tue Aug 2 09:49:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:49:30.212388 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20682f1c-072f-4395-b123-527a22eb5a81 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:49:30.264208 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:49:30.329367 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20682f1c-072f-4395-b123-527a22eb5a81 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:49:30.367930 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ade189-d484-4849-8fa7-3e472efec379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:49:30.369589 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ade189-d484-4849-8fa7-3e472efec379 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:30.369775 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ade189-d484-4849-8fa7-3e472efec379 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:30.370051 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ade189-d484-4849-8fa7-3e472efec379 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:30.382154 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ade189-d484-4849-8fa7-3e472efec379 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:49:30.384610 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ade189-d484-4849-8fa7-3e472efec379 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:49:30.406476 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-56ade189-d484-4849-8fa7-3e472efec379 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:49:30.406476 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:49:30.406476 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:49:30.406476 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:49:30.406476 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:49:30.406476 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:49:30.406476 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:49:30.406476 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:49:30.406476 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:49:30.406476 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:49:30.406476 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:49:30.406476 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:49:30.406476 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 09:49:30.406476 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 09:49:30.406476 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 09:49:30.408540 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 09:49:30.408540 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:49:30.408540 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:49:30.408540 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:49:30.408540 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:49:30.408540 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:49:30.408540 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:49:30.408540 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:49:30.408540 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:49:30.408540 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:49:30.408540 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:49:30.408540 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:49:30.408540 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:49:30.408540 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:49:30.408540 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:49:30.408540 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:49:30.408540 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:49:30.410282 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 09:49:30.410282 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 09:49:30.410282 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:49:30.410282 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:49:30.410282 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 604/1224] 10.222.0.64 () {56 vars in 1108 bytes} [Tue Aug 2 09:49:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 09:49:30.414690 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abdfdcc-0dd6-4c40-bafa-936a542777d9 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:49:30.556588 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abdfdcc-0dd6-4c40-bafa-936a542777d9 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:49:30.594507 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb2701d-ea34-44c9-9743-74e5d69c80f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:49:30.595371 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb2701d-ea34-44c9-9743-74e5d69c80f0 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:30.595667 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb2701d-ea34-44c9-9743-74e5d69c80f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:30.595937 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb2701d-ea34-44c9-9743-74e5d69c80f0 None admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:30.596694 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb2701d-ea34-44c9-9743-74e5d69c80f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:49:30.596991 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb2701d-ea34-44c9-9743-74e5d69c80f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:49:30.598435 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb2701d-ea34-44c9-9743-74e5d69c80f0 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:49:30.615017 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 621/1225] 10.222.0.64 () {56 vars in 1109 bytes} [Tue Aug 2 09:49:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:49:30.621335 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ade189-d484-4849-8fa7-3e472efec379 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:49:30.753547 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:49:30.761606 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ade189-d484-4849-8fa7-3e472efec379 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:49:30.797544 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38e3347-bdc6-4304-953f-f16200195a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:49:30.798188 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38e3347-bdc6-4304-953f-f16200195a8a None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:30.798475 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38e3347-bdc6-4304-953f-f16200195a8a None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:30.798751 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38e3347-bdc6-4304-953f-f16200195a8a None admin] PATH_INFO: `/v3/users/placement` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:30.806419 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38e3347-bdc6-4304-953f-f16200195a8a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:49:30.807943 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38e3347-bdc6-4304-953f-f16200195a8a None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:49:30.820156 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-e38e3347-bdc6-4304-953f-f16200195a8a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 02 09:49:30.820156 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:49:30.820156 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:49:30.820156 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:49:30.820156 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:49:30.820156 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:49:30.820156 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:49:30.820156 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:49:30.820156 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:49:30.820156 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:49:30.820156 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:49:30.820156 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:49:30.820156 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 09:49:30.820156 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 09:49:30.820156 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 09:49:30.820156 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 09:49:30.821867 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:49:30.821867 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:49:30.821867 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 09:49:30.821867 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 09:49:30.821867 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 09:49:30.821867 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 09:49:30.821867 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:49:30.821867 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:49:30.821867 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:49:30.821867 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:49:30.821867 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:49:30.821867 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:49:30.821867 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:49:30.821867 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:49:30.821867 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:49:30.821867 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:49:30.821867 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:49:30.821867 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:49:30.822854 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:49:30.822854 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:49:30.822854 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 09:49:30.822854 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 09:49:30.822854 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 09:49:30.822854 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 09:49:30.822854 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 09:49:30.822854 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 09:49:30.822854 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 02 09:49:30.822854 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:49:30.822854 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 605/1226] 10.222.0.64 () {56 vars in 1108 bytes} [Tue Aug 2 09:49:30 2022] GET /identity/v3/users/placement => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:49:30.832400 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb2701d-ea34-44c9-9743-74e5d69c80f0 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:49:30.889236 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:49:30.957340 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb2701d-ea34-44c9-9743-74e5d69c80f0 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:49:30.998773 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c162dcb5-f315-47da-9189-1ce7c4116c81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:49:30.999619 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c162dcb5-f315-47da-9189-1ce7c4116c81 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:30.999619 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c162dcb5-f315-47da-9189-1ce7c4116c81 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:30.999619 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c162dcb5-f315-47da-9189-1ce7c4116c81 None admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:31.000459 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c162dcb5-f315-47da-9189-1ce7c4116c81 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:49:31.000791 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c162dcb5-f315-47da-9189-1ce7c4116c81 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:49:31.002203 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c162dcb5-f315-47da-9189-1ce7c4116c81 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:49:31.020304 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 622/1227] 10.222.0.64 () {56 vars in 1143 bytes} [Tue Aug 2 09:49:30 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:49:31.026661 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38e3347-bdc6-4304-953f-f16200195a8a None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:49:31.162574 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38e3347-bdc6-4304-953f-f16200195a8a None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:49:31.202544 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e553d61-52d8-47f6-907f-c1c4e1176cd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:49:31.203187 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e553d61-52d8-47f6-907f-c1c4e1176cd8 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:31.203475 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e553d61-52d8-47f6-907f-c1c4e1176cd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:31.203753 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e553d61-52d8-47f6-907f-c1c4e1176cd8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:31.216638 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e553d61-52d8-47f6-907f-c1c4e1176cd8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:49:31.218098 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e553d61-52d8-47f6-907f-c1c4e1176cd8 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:49:31.237074 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-2e553d61-52d8-47f6-907f-c1c4e1176cd8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:49:31.237074 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:49:31.237074 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:49:31.237074 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:49:31.237074 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:49:31.237074 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:49:31.237074 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:49:31.237074 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:49:31.237074 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:49:31.237074 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:49:31.237074 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:49:31.237074 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:49:31.237074 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 09:49:31.237074 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 09:49:31.237074 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 09:49:31.238244 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 09:49:31.238244 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:49:31.238244 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:49:31.238244 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:49:31.238244 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:49:31.238244 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:49:31.238244 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:49:31.238244 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:49:31.238244 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:49:31.238244 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:49:31.238244 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:49:31.238244 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:49:31.238244 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:49:31.238244 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:49:31.238244 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:49:31.238244 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:49:31.238244 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:49:31.239309 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 09:49:31.239309 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 09:49:31.239309 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:49:31.239309 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:49:31.239309 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 606/1228] 10.222.0.64 () {56 vars in 1108 bytes} [Tue Aug 2 09:49:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:49:31.245041 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c162dcb5-f315-47da-9189-1ce7c4116c81 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:49:31.370704 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c162dcb5-f315-47da-9189-1ce7c4116c81 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:49:31.373817 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:49:31.416672 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc34309-208f-40c5-9a7a-20e204272dea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:49:31.417394 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc34309-208f-40c5-9a7a-20e204272dea None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:31.417783 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc34309-208f-40c5-9a7a-20e204272dea None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:31.418159 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc34309-208f-40c5-9a7a-20e204272dea None admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:31.419144 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc34309-208f-40c5-9a7a-20e204272dea None admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:49:31.419677 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc34309-208f-40c5-9a7a-20e204272dea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:49:31.422147 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc34309-208f-40c5-9a7a-20e204272dea None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:49:31.441158 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 623/1229] 10.222.0.64 () {56 vars in 1109 bytes} [Tue Aug 2 09:49:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:49:31.442749 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e553d61-52d8-47f6-907f-c1c4e1176cd8 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:49:31.453962 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:49:31.566680 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e553d61-52d8-47f6-907f-c1c4e1176cd8 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:49:31.608643 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4db31f-af53-4da4-a4cc-01378d66a4e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:49:31.609473 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4db31f-af53-4da4-a4cc-01378d66a4e0 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:31.609768 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4db31f-af53-4da4-a4cc-01378d66a4e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:31.610041 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4db31f-af53-4da4-a4cc-01378d66a4e0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:31.617787 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4db31f-af53-4da4-a4cc-01378d66a4e0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:49:31.620089 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4db31f-af53-4da4-a4cc-01378d66a4e0 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:49:31.633088 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-8c4db31f-af53-4da4-a4cc-01378d66a4e0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 09:49:31.633088 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:49:31.633088 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:49:31.633088 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:49:31.633088 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:49:31.633088 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:49:31.633088 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:49:31.633088 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:49:31.633088 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:49:31.633088 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:49:31.633088 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:49:31.633088 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:49:31.633088 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 09:49:31.633088 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 09:49:31.633088 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 09:49:31.635052 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 09:49:31.635052 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:49:31.635052 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:49:31.635052 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:49:31.635052 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:49:31.635052 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:49:31.635052 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:49:31.635052 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:49:31.635052 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:49:31.635052 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:49:31.635052 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:49:31.635052 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:49:31.635052 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:49:31.635052 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:49:31.635052 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:49:31.635052 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:49:31.635052 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:49:31.637083 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 09:49:31.637083 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 09:49:31.637083 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 09:49:31.637083 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 09:49:31.637083 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 09:49:31.637083 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 09:49:31.637083 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 09:49:31.637083 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:49:31.637083 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 607/1230] 10.222.0.64 () {56 vars in 1111 bytes} [Tue Aug 2 09:49:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:49:31.640711 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc34309-208f-40c5-9a7a-20e204272dea None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:49:31.768010 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc34309-208f-40c5-9a7a-20e204272dea None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:49:31.811334 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955ee72b-4e76-46f5-a81b-cf62c5601267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:49:31.812586 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955ee72b-4e76-46f5-a81b-cf62c5601267 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:31.813048 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955ee72b-4e76-46f5-a81b-cf62c5601267 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:31.813701 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955ee72b-4e76-46f5-a81b-cf62c5601267 None admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:31.815055 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955ee72b-4e76-46f5-a81b-cf62c5601267 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:49:31.815923 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955ee72b-4e76-46f5-a81b-cf62c5601267 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:49:31.818111 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955ee72b-4e76-46f5-a81b-cf62c5601267 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:49:31.831953 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 624/1231] 10.222.0.64 () {56 vars in 1148 bytes} [Tue Aug 2 09:49:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:49:31.839022 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4db31f-af53-4da4-a4cc-01378d66a4e0 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:49:31.917260 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:49:31.953776 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4db31f-af53-4da4-a4cc-01378d66a4e0 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:49:31.991516 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444086c2-fa53-45d3-b288-d61fd19653c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:49:31.992183 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444086c2-fa53-45d3-b288-d61fd19653c7 None admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:31.992495 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444086c2-fa53-45d3-b288-d61fd19653c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:31.992772 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444086c2-fa53-45d3-b288-d61fd19653c7 None admin] PATH_INFO: `/v3/projects/390bd7c4f91c4465b0c7ab0f9e436c4d/users/b770f073c86b4d88837a8b1f796a8b1d/roles/5704af09562a457fa35364e8af9481f1` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:32.008202 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444086c2-fa53-45d3-b288-d61fd19653c7 None admin] RBAC: Authorizing `identity:create_grant(project_id=390bd7c4f91c4465b0c7ab0f9e436c4d, user_id=b770f073c86b4d88837a8b1f796a8b1d, role_id=5704af09562a457fa35364e8af9481f1)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:49:32.009713 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444086c2-fa53-45d3-b288-d61fd19653c7 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:49:32.023062 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 608/1232] 10.222.0.64 () {58 vars in 1439 bytes} [Tue Aug 2 09:49:31 2022] PUT /identity/v3/projects/390bd7c4f91c4465b0c7ab0f9e436c4d/users/b770f073c86b4d88837a8b1f796a8b1d/roles/5704af09562a457fa35364e8af9481f1 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:49:34.109307 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5634e162-9db6-4478-a720-ca06fe6d80f4 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:34.109897 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5634e162-9db6-4478-a720-ca06fe6d80f4 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:34.109897 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5634e162-9db6-4478-a720-ca06fe6d80f4 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:34.111223 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 625/1233] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:49:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:49:34.116684 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace9e484-c5a2-4fc1-81f5-4b49ac54d503 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:34.117125 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace9e484-c5a2-4fc1-81f5-4b49ac54d503 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:34.117273 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace9e484-c5a2-4fc1-81f5-4b49ac54d503 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:34.142872 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-ace9e484-c5a2-4fc1-81f5-4b49ac54d503 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:49:34.144985 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-ace9e484-c5a2-4fc1-81f5-4b49ac54d503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:49:34.223541 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 609/1234] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:49:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 09:49:34.248494 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5634e162-9db6-4478-a720-ca06fe6d80f4 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:49:34.277081 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:49:34.358882 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5634e162-9db6-4478-a720-ca06fe6d80f4 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:49:34.404736 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906e8835-68bf-4856-85d8-33425c674680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:49:34.405712 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906e8835-68bf-4856-85d8-33425c674680 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:34.406200 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906e8835-68bf-4856-85d8-33425c674680 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:34.406676 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906e8835-68bf-4856-85d8-33425c674680 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:34.419600 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906e8835-68bf-4856-85d8-33425c674680 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 09:49:34.425082 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-906e8835-68bf-4856-85d8-33425c674680 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 09:49:34.425082 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:49:34.425082 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:49:34.425082 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:49:34.425082 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:49:34.425082 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:49:34.425082 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:49:34.425082 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:49:34.425082 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:49:34.425082 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:49:34.425082 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:49:34.425082 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:49:34.425082 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 09:49:34.425082 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 09:49:34.425082 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 09:49:34.425082 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 09:49:34.426245 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 09:49:34.426245 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 09:49:34.426245 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 09:49:34.426245 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 09:49:34.426245 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:49:34.426245 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:49:34.426245 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:49:34.426245 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:49:34.426245 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:49:34.426245 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:49:34.426245 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:49:34.426245 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:49:34.426245 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:49:34.426245 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:49:34.426245 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:49:34.426245 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:49:34.427264 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:49:34.427264 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:49:34.427264 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:49:34.427264 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:49:34.427264 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 09:49:34.427264 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 09:49:34.427264 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 09:49:34.427264 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 09:49:34.427264 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 09:49:34.427264 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 09:49:34.427264 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 09:49:34.427264 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:49:34.427264 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 626/1235] 10.222.0.64 () {56 vars in 1102 bytes} [Tue Aug 2 09:49:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:49:34.434500 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace9e484-c5a2-4fc1-81f5-4b49ac54d503 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:49:34.481519 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:49:34.535751 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace9e484-c5a2-4fc1-81f5-4b49ac54d503 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:49:34.571310 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb40c6b-cd62-4eb0-96a7-8b26bbdb351b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:49:34.571936 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb40c6b-cd62-4eb0-96a7-8b26bbdb351b None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:34.572219 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb40c6b-cd62-4eb0-96a7-8b26bbdb351b None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:34.572518 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb40c6b-cd62-4eb0-96a7-8b26bbdb351b None admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:34.573244 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb40c6b-cd62-4eb0-96a7-8b26bbdb351b None admin] RBAC: Adding query filter params (name=service) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:49:34.573550 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb40c6b-cd62-4eb0-96a7-8b26bbdb351b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:49:34.574937 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb40c6b-cd62-4eb0-96a7-8b26bbdb351b None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:49:34.583832 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 610/1236] 10.222.0.64 () {56 vars in 1103 bytes} [Tue Aug 2 09:49:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:49:34.589656 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906e8835-68bf-4856-85d8-33425c674680 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:49:34.679156 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:49:34.696480 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906e8835-68bf-4856-85d8-33425c674680 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:49:34.733164 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf1f7d6-d147-4edf-82ba-097417039c12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:49:34.733808 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf1f7d6-d147-4edf-82ba-097417039c12 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:34.734135 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf1f7d6-d147-4edf-82ba-097417039c12 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:34.734437 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf1f7d6-d147-4edf-82ba-097417039c12 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:34.746723 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf1f7d6-d147-4edf-82ba-097417039c12 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:49:34.748379 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf1f7d6-d147-4edf-82ba-097417039c12 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:49:34.764891 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-edf1f7d6-d147-4edf-82ba-097417039c12 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:49:34.764891 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:49:34.764891 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:49:34.764891 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:49:34.764891 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:49:34.764891 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:49:34.764891 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:49:34.764891 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:49:34.764891 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:49:34.764891 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:49:34.764891 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:49:34.764891 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:49:34.764891 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 09:49:34.764891 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 09:49:34.764891 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 09:49:34.765916 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 09:49:34.765916 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:49:34.765916 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:49:34.765916 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:49:34.765916 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:49:34.765916 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:49:34.765916 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:49:34.765916 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:49:34.765916 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:49:34.765916 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:49:34.765916 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:49:34.765916 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:49:34.765916 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:49:34.765916 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:49:34.765916 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:49:34.765916 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:49:34.765916 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:49:34.766888 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 09:49:34.766888 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 09:49:34.766888 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:49:34.766888 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:49:34.771153 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 627/1237] 10.222.0.64 () {56 vars in 1108 bytes} [Tue Aug 2 09:49:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:49:34.772277 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb40c6b-cd62-4eb0-96a7-8b26bbdb351b None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:49:34.880200 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb40c6b-cd62-4eb0-96a7-8b26bbdb351b None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:49:34.891150 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:49:34.924287 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01c409a-ff81-49de-9692-6f410a9a5d74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:49:34.924287 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01c409a-ff81-49de-9692-6f410a9a5d74 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:34.924287 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01c409a-ff81-49de-9692-6f410a9a5d74 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:34.924728 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01c409a-ff81-49de-9692-6f410a9a5d74 None admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:34.925095 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01c409a-ff81-49de-9692-6f410a9a5d74 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:49:34.925408 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01c409a-ff81-49de-9692-6f410a9a5d74 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:49:34.926785 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01c409a-ff81-49de-9692-6f410a9a5d74 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:49:34.941210 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 611/1238] 10.222.0.64 () {56 vars in 1109 bytes} [Tue Aug 2 09:49:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:49:34.948352 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf1f7d6-d147-4edf-82ba-097417039c12 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:49:35.065519 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf1f7d6-d147-4edf-82ba-097417039c12 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:49:35.113557 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb137d9-efef-4fb6-925f-b6d26154b487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:49:35.114387 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb137d9-efef-4fb6-925f-b6d26154b487 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:35.114750 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb137d9-efef-4fb6-925f-b6d26154b487 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:35.115263 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb137d9-efef-4fb6-925f-b6d26154b487 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:35.120320 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:49:35.129291 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb137d9-efef-4fb6-925f-b6d26154b487 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:49:35.131930 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb137d9-efef-4fb6-925f-b6d26154b487 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:49:35.151721 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-2eb137d9-efef-4fb6-925f-b6d26154b487 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 02 09:49:35.151721 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:49:35.151721 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:49:35.151721 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:49:35.151721 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:49:35.151721 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:49:35.151721 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:49:35.151721 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:49:35.151721 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:49:35.151721 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:49:35.151721 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:49:35.151721 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:49:35.151721 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 09:49:35.151721 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 09:49:35.151721 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 09:49:35.151721 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 09:49:35.153840 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:49:35.153840 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:49:35.153840 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 09:49:35.153840 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 09:49:35.153840 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 09:49:35.153840 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 09:49:35.153840 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:49:35.153840 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:49:35.153840 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:49:35.153840 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:49:35.153840 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:49:35.153840 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:49:35.153840 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:49:35.153840 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:49:35.153840 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:49:35.153840 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:49:35.153840 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:49:35.153840 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:49:35.155712 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:49:35.155712 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:49:35.155712 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 09:49:35.155712 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 09:49:35.155712 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 09:49:35.155712 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 09:49:35.155712 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 09:49:35.155712 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 09:49:35.155712 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 02 09:49:35.155712 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:49:35.155712 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 628/1239] 10.222.0.64 () {56 vars in 1108 bytes} [Tue Aug 2 09:49:34 2022] GET /identity/v3/users/placement => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:49:35.162954 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01c409a-ff81-49de-9692-6f410a9a5d74 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:49:35.290846 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01c409a-ff81-49de-9692-6f410a9a5d74 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:49:35.330483 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae41ff1-94d6-4aa6-a6c8-1a2cdcd611be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:49:35.331111 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae41ff1-94d6-4aa6-a6c8-1a2cdcd611be None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:35.331392 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae41ff1-94d6-4aa6-a6c8-1a2cdcd611be None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:35.331665 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae41ff1-94d6-4aa6-a6c8-1a2cdcd611be None admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:35.332515 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae41ff1-94d6-4aa6-a6c8-1a2cdcd611be None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:49:35.332820 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae41ff1-94d6-4aa6-a6c8-1a2cdcd611be None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:49:35.334253 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae41ff1-94d6-4aa6-a6c8-1a2cdcd611be None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:49:35.353747 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 612/1240] 10.222.0.64 () {56 vars in 1143 bytes} [Tue Aug 2 09:49:35 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:49:35.362997 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb137d9-efef-4fb6-925f-b6d26154b487 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:49:35.501864 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb137d9-efef-4fb6-925f-b6d26154b487 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:49:35.551596 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee930e1a-1978-4b88-b713-51d1efcffeee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:49:35.552142 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee930e1a-1978-4b88-b713-51d1efcffeee None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:35.552634 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee930e1a-1978-4b88-b713-51d1efcffeee None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:35.552890 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee930e1a-1978-4b88-b713-51d1efcffeee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:35.623208 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:49:35.630357 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee930e1a-1978-4b88-b713-51d1efcffeee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:49:35.631944 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee930e1a-1978-4b88-b713-51d1efcffeee None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:49:35.742365 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 629/1241] 10.222.0.64 () {58 vars in 1282 bytes} [Tue Aug 2 09:49:35 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 02 09:49:35.750180 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae41ff1-94d6-4aa6-a6c8-1a2cdcd611be None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:49:35.785377 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:49:35.866941 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae41ff1-94d6-4aa6-a6c8-1a2cdcd611be None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:49:35.910221 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87e1ae8-07b8-4363-9011-a2b63a375e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:49:35.911065 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87e1ae8-07b8-4363-9011-a2b63a375e7b None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:35.911476 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87e1ae8-07b8-4363-9011-a2b63a375e7b None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:35.911854 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87e1ae8-07b8-4363-9011-a2b63a375e7b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:35.925197 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87e1ae8-07b8-4363-9011-a2b63a375e7b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:49:35.927167 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87e1ae8-07b8-4363-9011-a2b63a375e7b None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:49:35.949329 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-c87e1ae8-07b8-4363-9011-a2b63a375e7b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:49:35.949329 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:49:35.949329 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:49:35.949329 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:49:35.949329 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:49:35.949329 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:49:35.949329 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:49:35.949329 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:49:35.949329 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:49:35.949329 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:49:35.949329 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:49:35.949329 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:49:35.949329 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 09:49:35.949329 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 09:49:35.949329 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 09:49:35.951201 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 09:49:35.951201 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:49:35.951201 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:49:35.951201 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:49:35.951201 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:49:35.951201 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:49:35.951201 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:49:35.951201 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:49:35.951201 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:49:35.951201 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:49:35.951201 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:49:35.951201 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:49:35.951201 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:49:35.951201 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:49:35.951201 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:49:35.951201 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:49:35.951201 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:49:35.953139 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 09:49:35.953139 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 09:49:35.953139 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:49:35.953139 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:49:35.953139 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 613/1242] 10.222.0.64 () {56 vars in 1108 bytes} [Tue Aug 2 09:49:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:49:35.957229 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee930e1a-1978-4b88-b713-51d1efcffeee None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:49:36.100100 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:49:36.107126 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee930e1a-1978-4b88-b713-51d1efcffeee None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:49:36.145923 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3f6afe-7b9f-40a1-88d6-854ba1c14900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:49:36.146599 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3f6afe-7b9f-40a1-88d6-854ba1c14900 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:36.146785 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3f6afe-7b9f-40a1-88d6-854ba1c14900 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:36.147109 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3f6afe-7b9f-40a1-88d6-854ba1c14900 None admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:36.147836 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3f6afe-7b9f-40a1-88d6-854ba1c14900 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:49:36.148210 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3f6afe-7b9f-40a1-88d6-854ba1c14900 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:49:36.149943 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3f6afe-7b9f-40a1-88d6-854ba1c14900 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:49:36.163323 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 630/1243] 10.222.0.64 () {56 vars in 1109 bytes} [Tue Aug 2 09:49:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:49:36.169578 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87e1ae8-07b8-4363-9011-a2b63a375e7b None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:49:36.264501 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:49:36.290327 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87e1ae8-07b8-4363-9011-a2b63a375e7b None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:49:36.327684 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24794ff4-44ae-4c1e-951d-d2c2d9368464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:49:36.328373 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24794ff4-44ae-4c1e-951d-d2c2d9368464 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:36.328688 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24794ff4-44ae-4c1e-951d-d2c2d9368464 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:36.328965 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24794ff4-44ae-4c1e-951d-d2c2d9368464 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:36.337250 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24794ff4-44ae-4c1e-951d-d2c2d9368464 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:49:36.338867 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24794ff4-44ae-4c1e-951d-d2c2d9368464 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:49:36.351541 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-24794ff4-44ae-4c1e-951d-d2c2d9368464 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 09:49:36.351541 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:49:36.351541 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:49:36.351541 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:49:36.351541 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:49:36.351541 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:49:36.351541 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:49:36.351541 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:49:36.351541 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:49:36.351541 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:49:36.351541 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:49:36.351541 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:49:36.351541 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 09:49:36.351541 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 09:49:36.351541 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 09:49:36.353346 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 09:49:36.353346 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:49:36.353346 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:49:36.353346 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:49:36.353346 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:49:36.353346 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:49:36.353346 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:49:36.353346 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:49:36.353346 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:49:36.353346 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:49:36.353346 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:49:36.353346 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:49:36.353346 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:49:36.353346 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:49:36.353346 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:49:36.353346 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:49:36.353346 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:49:36.354660 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 09:49:36.354660 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 09:49:36.354660 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 09:49:36.354660 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 09:49:36.354660 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 09:49:36.354660 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 09:49:36.354660 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 09:49:36.354660 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:49:36.354660 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 614/1244] 10.222.0.64 () {56 vars in 1111 bytes} [Tue Aug 2 09:49:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:49:36.359466 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3f6afe-7b9f-40a1-88d6-854ba1c14900 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:49:36.478407 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3f6afe-7b9f-40a1-88d6-854ba1c14900 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:49:36.520619 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f44e83-ec2d-4f14-b7f5-0a04dbeea878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:49:36.521254 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f44e83-ec2d-4f14-b7f5-0a04dbeea878 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:36.521537 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f44e83-ec2d-4f14-b7f5-0a04dbeea878 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:36.521809 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f44e83-ec2d-4f14-b7f5-0a04dbeea878 None admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:36.522563 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f44e83-ec2d-4f14-b7f5-0a04dbeea878 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:49:36.522873 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f44e83-ec2d-4f14-b7f5-0a04dbeea878 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:49:36.525012 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f44e83-ec2d-4f14-b7f5-0a04dbeea878 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:49:36.537833 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 631/1245] 10.222.0.64 () {56 vars in 1148 bytes} [Tue Aug 2 09:49:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:49:36.545066 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24794ff4-44ae-4c1e-951d-d2c2d9368464 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:49:36.672584 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24794ff4-44ae-4c1e-951d-d2c2d9368464 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:49:36.710701 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04722fdc-9d98-4daf-9785-21e59a7eafd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:49:36.711335 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04722fdc-9d98-4daf-9785-21e59a7eafd2 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:36.712120 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04722fdc-9d98-4daf-9785-21e59a7eafd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:36.712505 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04722fdc-9d98-4daf-9785-21e59a7eafd2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:36.713374 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04722fdc-9d98-4daf-9785-21e59a7eafd2 None admin] RBAC: Adding query filter params (role.id=5704af09562a457fa35364e8af9481f1, scope.project.id=390bd7c4f91c4465b0c7ab0f9e436c4d, user.id=b770f073c86b4d88837a8b1f796a8b1d) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:49:36.713711 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04722fdc-9d98-4daf-9785-21e59a7eafd2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:49:36.715340 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04722fdc-9d98-4daf-9785-21e59a7eafd2 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:49:36.722339 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 615/1246] 10.222.0.64 () {56 vars in 1374 bytes} [Tue Aug 2 09:49:36 2022] GET /identity/v3/role_assignments?user.id=b770f073c86b4d88837a8b1f796a8b1d&scope.project.id=390bd7c4f91c4465b0c7ab0f9e436c4d&role.id=5704af09562a457fa35364e8af9481f1 => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:49:38.809424 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d6fc2d-1159-47cd-9cee-33483315a8ed None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:38.810032 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d6fc2d-1159-47cd-9cee-33483315a8ed None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:38.810032 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d6fc2d-1159-47cd-9cee-33483315a8ed None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:38.811338 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 632/1247] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:49:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:49:38.817379 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b00c32c-4262-422f-a488-5b7869029fc8 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:38.817796 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b00c32c-4262-422f-a488-5b7869029fc8 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:38.818012 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b00c32c-4262-422f-a488-5b7869029fc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:38.825148 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:49:38.838716 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-0b00c32c-4262-422f-a488-5b7869029fc8 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:49:38.840086 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-0b00c32c-4262-422f-a488-5b7869029fc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:49:38.918747 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 616/1248] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:49:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 02 09:49:38.947386 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d6fc2d-1159-47cd-9cee-33483315a8ed None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:49:38.967143 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:49:39.088873 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d6fc2d-1159-47cd-9cee-33483315a8ed None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:49:39.143691 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b022f0-82cb-4429-b2fa-1f04133fd056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:49:39.144383 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b022f0-82cb-4429-b2fa-1f04133fd056 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:39.144689 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b022f0-82cb-4429-b2fa-1f04133fd056 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:39.144959 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b022f0-82cb-4429-b2fa-1f04133fd056 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:39.157352 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b022f0-82cb-4429-b2fa-1f04133fd056 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 09:49:39.163392 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-58b022f0-82cb-4429-b2fa-1f04133fd056 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 09:49:39.163392 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:49:39.163392 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:49:39.163392 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:49:39.163392 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:49:39.163392 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:49:39.163392 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:49:39.163392 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:49:39.163392 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:49:39.163392 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:49:39.163392 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:49:39.163392 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:49:39.163392 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 09:49:39.163392 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 09:49:39.163392 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 09:49:39.163392 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 09:49:39.165118 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 09:49:39.165118 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 09:49:39.165118 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 09:49:39.165118 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 09:49:39.165118 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:49:39.165118 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:49:39.165118 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:49:39.165118 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:49:39.165118 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:49:39.165118 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:49:39.165118 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:49:39.165118 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:49:39.165118 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:49:39.165118 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:49:39.165118 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:49:39.165118 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:49:39.166113 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:49:39.166113 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:49:39.166113 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:49:39.166113 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:49:39.166113 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 09:49:39.166113 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 09:49:39.166113 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 09:49:39.166113 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 09:49:39.166113 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 09:49:39.166113 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 09:49:39.166113 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 09:49:39.166113 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:49:39.166113 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 633/1249] 10.222.0.64 () {56 vars in 1096 bytes} [Tue Aug 2 09:49:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:49:39.173561 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b00c32c-4262-422f-a488-5b7869029fc8 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:49:39.314280 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b00c32c-4262-422f-a488-5b7869029fc8 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:49:39.348090 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:49:39.356632 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63d120d-6201-4324-9356-8420310b1b3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:49:39.357265 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63d120d-6201-4324-9356-8420310b1b3a None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:39.357573 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63d120d-6201-4324-9356-8420310b1b3a None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:39.357850 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63d120d-6201-4324-9356-8420310b1b3a None admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:39.358621 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63d120d-6201-4324-9356-8420310b1b3a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:49:39.358893 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63d120d-6201-4324-9356-8420310b1b3a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:49:39.360615 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63d120d-6201-4324-9356-8420310b1b3a None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:49:39.371108 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 617/1250] 10.222.0.64 () {56 vars in 1099 bytes} [Tue Aug 2 09:49:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:49:39.378024 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b022f0-82cb-4429-b2fa-1f04133fd056 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:49:39.500733 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:49:39.529294 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b022f0-82cb-4429-b2fa-1f04133fd056 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:49:39.567500 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91a82e3-bbc3-44ca-89bc-da166216001d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:49:39.568201 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91a82e3-bbc3-44ca-89bc-da166216001d None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:39.568576 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91a82e3-bbc3-44ca-89bc-da166216001d None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:39.568921 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91a82e3-bbc3-44ca-89bc-da166216001d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:39.581735 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91a82e3-bbc3-44ca-89bc-da166216001d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:49:39.583426 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91a82e3-bbc3-44ca-89bc-da166216001d None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:49:39.610572 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-e91a82e3-bbc3-44ca-89bc-da166216001d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:49:39.610572 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:49:39.610572 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:49:39.610572 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:49:39.610572 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:49:39.610572 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:49:39.610572 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:49:39.610572 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:49:39.610572 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:49:39.610572 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:49:39.610572 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:49:39.610572 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:49:39.610572 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 09:49:39.610572 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 09:49:39.610572 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 09:49:39.611558 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 09:49:39.611558 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:49:39.611558 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:49:39.611558 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:49:39.611558 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:49:39.611558 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:49:39.611558 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:49:39.611558 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:49:39.611558 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:49:39.611558 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:49:39.611558 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:49:39.611558 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:49:39.611558 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:49:39.611558 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:49:39.611558 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:49:39.611558 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:49:39.611558 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:49:39.612505 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 09:49:39.612505 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 09:49:39.612505 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:49:39.612505 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:49:39.612505 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 634/1251] 10.222.0.64 () {56 vars in 1108 bytes} [Tue Aug 2 09:49:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:49:39.612818 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63d120d-6201-4324-9356-8420310b1b3a None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:49:39.729089 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63d120d-6201-4324-9356-8420310b1b3a None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:49:39.773883 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a44522-5aa4-4140-aff2-b7e880bddbb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:49:39.775228 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a44522-5aa4-4140-aff2-b7e880bddbb3 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:39.775552 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a44522-5aa4-4140-aff2-b7e880bddbb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:39.775862 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a44522-5aa4-4140-aff2-b7e880bddbb3 None admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:39.776876 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a44522-5aa4-4140-aff2-b7e880bddbb3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:49:39.777193 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a44522-5aa4-4140-aff2-b7e880bddbb3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:49:39.779200 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a44522-5aa4-4140-aff2-b7e880bddbb3 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:49:39.795239 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 618/1252] 10.222.0.64 () {56 vars in 1109 bytes} [Tue Aug 2 09:49:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:49:39.801701 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91a82e3-bbc3-44ca-89bc-da166216001d None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:49:39.917846 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91a82e3-bbc3-44ca-89bc-da166216001d None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:49:39.951866 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:49:39.963619 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fed9a82-5cc9-4d4e-86b4-d3a44766760d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:49:39.964490 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fed9a82-5cc9-4d4e-86b4-d3a44766760d None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:39.964860 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fed9a82-5cc9-4d4e-86b4-d3a44766760d None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:39.965143 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fed9a82-5cc9-4d4e-86b4-d3a44766760d None admin] PATH_INFO: `/v3/users/placement` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:39.973854 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fed9a82-5cc9-4d4e-86b4-d3a44766760d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:49:39.975297 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fed9a82-5cc9-4d4e-86b4-d3a44766760d None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:49:39.988593 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-0fed9a82-5cc9-4d4e-86b4-d3a44766760d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 02 09:49:39.988593 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:49:39.988593 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:49:39.988593 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:49:39.988593 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:49:39.988593 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:49:39.988593 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:49:39.988593 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:49:39.988593 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:49:39.988593 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:49:39.988593 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:49:39.988593 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:49:39.988593 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 09:49:39.988593 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 09:49:39.988593 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 09:49:39.988593 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 09:49:39.990028 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:49:39.990028 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:49:39.990028 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 09:49:39.990028 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 09:49:39.990028 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 09:49:39.990028 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 09:49:39.990028 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:49:39.990028 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:49:39.990028 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:49:39.990028 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:49:39.990028 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:49:39.990028 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:49:39.990028 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:49:39.990028 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:49:39.990028 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:49:39.990028 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:49:39.990028 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:49:39.990028 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:49:39.991138 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:49:39.991138 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:49:39.991138 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 09:49:39.991138 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 09:49:39.991138 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 09:49:39.991138 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 09:49:39.991138 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 09:49:39.991138 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 09:49:39.991138 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 02 09:49:39.991138 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:49:39.991138 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 635/1253] 10.222.0.64 () {56 vars in 1108 bytes} [Tue Aug 2 09:49:39 2022] GET /identity/v3/users/placement => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:49:39.995974 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a44522-5aa4-4140-aff2-b7e880bddbb3 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:49:40.059014 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:49:40.115321 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a44522-5aa4-4140-aff2-b7e880bddbb3 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:49:40.160632 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4aa777e-840d-4b6d-aca4-6a16c11e7ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:49:40.161172 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aa777e-840d-4b6d-aca4-6a16c11e7ad2 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:40.161498 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aa777e-840d-4b6d-aca4-6a16c11e7ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:40.161815 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aa777e-840d-4b6d-aca4-6a16c11e7ad2 None admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:40.162580 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4aa777e-840d-4b6d-aca4-6a16c11e7ad2 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:49:40.162919 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4aa777e-840d-4b6d-aca4-6a16c11e7ad2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:49:40.164463 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4aa777e-840d-4b6d-aca4-6a16c11e7ad2 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:49:40.183636 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 619/1254] 10.222.0.64 () {56 vars in 1143 bytes} [Tue Aug 2 09:49:39 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:49:40.189720 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fed9a82-5cc9-4d4e-86b4-d3a44766760d None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:49:40.313542 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fed9a82-5cc9-4d4e-86b4-d3a44766760d None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:49:40.360766 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b15381-7b5c-461b-ba1f-8aee9f2d9a48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:49:40.361723 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b15381-7b5c-461b-ba1f-8aee9f2d9a48 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:40.362161 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b15381-7b5c-461b-ba1f-8aee9f2d9a48 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:40.362587 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b15381-7b5c-461b-ba1f-8aee9f2d9a48 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:40.433736 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:49:40.448050 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b15381-7b5c-461b-ba1f-8aee9f2d9a48 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:49:40.449585 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b15381-7b5c-461b-ba1f-8aee9f2d9a48 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:49:40.569521 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 636/1255] 10.222.0.64 () {58 vars in 1282 bytes} [Tue Aug 2 09:49:40 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 02 09:49:40.576352 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4aa777e-840d-4b6d-aca4-6a16c11e7ad2 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:49:40.698730 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4aa777e-840d-4b6d-aca4-6a16c11e7ad2 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:49:40.723614 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:49:40.751423 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d754a7-310d-41fc-bbb5-79e50ffb4ca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:49:40.752208 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d754a7-310d-41fc-bbb5-79e50ffb4ca2 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:40.757586 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d754a7-310d-41fc-bbb5-79e50ffb4ca2 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:40.757586 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d754a7-310d-41fc-bbb5-79e50ffb4ca2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:40.772341 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d754a7-310d-41fc-bbb5-79e50ffb4ca2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:49:40.773810 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d754a7-310d-41fc-bbb5-79e50ffb4ca2 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:49:40.794613 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-b2d754a7-310d-41fc-bbb5-79e50ffb4ca2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:49:40.794613 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:49:40.794613 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:49:40.794613 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:49:40.794613 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:49:40.794613 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:49:40.794613 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:49:40.794613 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:49:40.794613 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:49:40.794613 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:49:40.794613 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:49:40.794613 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:49:40.794613 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 09:49:40.794613 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 09:49:40.794613 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 09:49:40.795719 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 09:49:40.795719 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:49:40.795719 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:49:40.795719 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:49:40.795719 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:49:40.795719 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:49:40.795719 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:49:40.795719 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:49:40.795719 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:49:40.795719 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:49:40.795719 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:49:40.795719 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:49:40.795719 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:49:40.795719 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:49:40.795719 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:49:40.795719 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:49:40.795719 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:49:40.796698 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 09:49:40.796698 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 09:49:40.796698 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:49:40.796698 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:49:40.801331 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 620/1256] 10.222.0.64 () {56 vars in 1108 bytes} [Tue Aug 2 09:49:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:49:40.802320 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b15381-7b5c-461b-ba1f-8aee9f2d9a48 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:49:40.911913 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:49:40.925179 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b15381-7b5c-461b-ba1f-8aee9f2d9a48 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:49:40.963846 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689ca86d-27c5-41a0-857a-d99d759e2667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:49:40.964396 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689ca86d-27c5-41a0-857a-d99d759e2667 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:40.964680 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689ca86d-27c5-41a0-857a-d99d759e2667 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:40.964964 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689ca86d-27c5-41a0-857a-d99d759e2667 None admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:40.965686 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689ca86d-27c5-41a0-857a-d99d759e2667 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:49:40.965951 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689ca86d-27c5-41a0-857a-d99d759e2667 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:49:40.967381 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689ca86d-27c5-41a0-857a-d99d759e2667 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:49:40.987707 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 637/1257] 10.222.0.64 () {56 vars in 1109 bytes} [Tue Aug 2 09:49:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:49:40.993593 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d754a7-310d-41fc-bbb5-79e50ffb4ca2 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:49:41.114685 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d754a7-310d-41fc-bbb5-79e50ffb4ca2 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:49:41.150473 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be98b9f8-baad-481a-9918-f693a3c57ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:49:41.151151 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be98b9f8-baad-481a-9918-f693a3c57ec7 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:41.151402 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be98b9f8-baad-481a-9918-f693a3c57ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:41.151680 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be98b9f8-baad-481a-9918-f693a3c57ec7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:41.159310 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be98b9f8-baad-481a-9918-f693a3c57ec7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:49:41.160925 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be98b9f8-baad-481a-9918-f693a3c57ec7 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:49:41.174844 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-be98b9f8-baad-481a-9918-f693a3c57ec7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 09:49:41.174844 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:49:41.174844 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:49:41.174844 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:49:41.174844 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:49:41.174844 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:49:41.174844 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:49:41.174844 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:49:41.174844 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:49:41.174844 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:49:41.174844 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:49:41.174844 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:49:41.174844 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 09:49:41.174844 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 09:49:41.174844 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 09:49:41.176355 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 09:49:41.176355 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:49:41.176355 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:49:41.176355 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:49:41.176355 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:49:41.176355 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:49:41.176355 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:49:41.176355 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:49:41.176355 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:49:41.176355 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:49:41.176355 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:49:41.176355 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:49:41.176355 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:49:41.176355 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:49:41.176355 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:49:41.176355 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:49:41.176355 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:49:41.177576 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 09:49:41.177576 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 09:49:41.177576 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 09:49:41.177576 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 09:49:41.177576 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 09:49:41.177576 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 09:49:41.177576 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 09:49:41.177576 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:49:41.177576 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 621/1258] 10.222.0.64 () {56 vars in 1111 bytes} [Tue Aug 2 09:49:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:49:41.182452 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689ca86d-27c5-41a0-857a-d99d759e2667 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:49:41.311738 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689ca86d-27c5-41a0-857a-d99d759e2667 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:49:41.364226 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e604c37c-33a8-4003-823f-dfa8d42b394f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:49:41.365093 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e604c37c-33a8-4003-823f-dfa8d42b394f None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:41.365385 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e604c37c-33a8-4003-823f-dfa8d42b394f None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:41.365668 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e604c37c-33a8-4003-823f-dfa8d42b394f None admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:41.366609 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e604c37c-33a8-4003-823f-dfa8d42b394f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:49:41.366948 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e604c37c-33a8-4003-823f-dfa8d42b394f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:49:41.369024 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e604c37c-33a8-4003-823f-dfa8d42b394f None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:49:41.385542 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 638/1259] 10.222.0.64 () {56 vars in 1148 bytes} [Tue Aug 2 09:49:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:49:41.393491 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be98b9f8-baad-481a-9918-f693a3c57ec7 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:49:41.424682 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:49:41.506970 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be98b9f8-baad-481a-9918-f693a3c57ec7 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:49:41.543548 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c498e5-9129-4495-9099-d7130dadcbde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:49:41.544164 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c498e5-9129-4495-9099-d7130dadcbde None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:41.544469 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c498e5-9129-4495-9099-d7130dadcbde None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:41.544746 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c498e5-9129-4495-9099-d7130dadcbde None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:41.545598 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c498e5-9129-4495-9099-d7130dadcbde None admin] RBAC: Adding query filter params (role.id=1000cf318fa14915b8091599eaead5de, scope.project.id=390bd7c4f91c4465b0c7ab0f9e436c4d, user.id=b770f073c86b4d88837a8b1f796a8b1d) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:49:41.545843 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c498e5-9129-4495-9099-d7130dadcbde None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:49:41.547282 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c498e5-9129-4495-9099-d7130dadcbde None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:49:41.553071 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 622/1260] 10.222.0.64 () {56 vars in 1374 bytes} [Tue Aug 2 09:49:41 2022] GET /identity/v3/role_assignments?user.id=b770f073c86b4d88837a8b1f796a8b1d&scope.project.id=390bd7c4f91c4465b0c7ab0f9e436c4d&role.id=1000cf318fa14915b8091599eaead5de => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:49:43.597638 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8699635f-7028-4145-9248-5afc45129022 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:43.598236 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8699635f-7028-4145-9248-5afc45129022 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:43.598236 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8699635f-7028-4145-9248-5afc45129022 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:43.599517 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 639/1261] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:49:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:49:43.604833 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5427a612-2730-437d-90f0-98781a62a1e4 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:43.605287 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5427a612-2730-437d-90f0-98781a62a1e4 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:43.605388 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5427a612-2730-437d-90f0-98781a62a1e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:43.624039 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-5427a612-2730-437d-90f0-98781a62a1e4 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:49:43.625269 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-5427a612-2730-437d-90f0-98781a62a1e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:49:43.694478 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 623/1262] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:49:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 09:49:43.717571 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8699635f-7028-4145-9248-5afc45129022 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:49:43.729416 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:49:43.819220 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8699635f-7028-4145-9248-5afc45129022 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:49:43.855693 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8160ba52-aa46-4755-a2f1-32042f37de63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:49:43.856271 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8160ba52-aa46-4755-a2f1-32042f37de63 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:43.856575 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8160ba52-aa46-4755-a2f1-32042f37de63 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:43.856853 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8160ba52-aa46-4755-a2f1-32042f37de63 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:43.866698 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8160ba52-aa46-4755-a2f1-32042f37de63 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 09:49:43.872047 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-8160ba52-aa46-4755-a2f1-32042f37de63 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 09:49:43.872047 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:49:43.872047 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:49:43.872047 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:49:43.872047 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:49:43.872047 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:49:43.872047 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:49:43.872047 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:49:43.872047 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:49:43.872047 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:49:43.872047 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:49:43.872047 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:49:43.872047 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 09:49:43.872047 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 09:49:43.872047 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 09:49:43.872047 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 09:49:43.873252 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 09:49:43.873252 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 09:49:43.873252 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 09:49:43.873252 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 09:49:43.873252 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:49:43.873252 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:49:43.873252 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:49:43.873252 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:49:43.873252 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:49:43.873252 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:49:43.873252 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:49:43.873252 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:49:43.873252 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:49:43.873252 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:49:43.873252 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:49:43.873252 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:49:43.874397 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:49:43.874397 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:49:43.874397 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:49:43.874397 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:49:43.874397 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 09:49:43.874397 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 09:49:43.874397 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 09:49:43.874397 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 09:49:43.874397 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 09:49:43.874397 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 09:49:43.874397 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 09:49:43.874397 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:49:43.874397 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 640/1263] 10.222.0.64 () {56 vars in 1096 bytes} [Tue Aug 2 09:49:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:49:43.879225 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5427a612-2730-437d-90f0-98781a62a1e4 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:49:43.897675 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:49:43.974911 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5427a612-2730-437d-90f0-98781a62a1e4 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:49:44.007161 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bab6bc5-8aa1-4e27-9bea-bf515566a7fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:49:44.007767 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bab6bc5-8aa1-4e27-9bea-bf515566a7fe None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:44.008049 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bab6bc5-8aa1-4e27-9bea-bf515566a7fe None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:44.008883 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bab6bc5-8aa1-4e27-9bea-bf515566a7fe None admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:44.009591 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bab6bc5-8aa1-4e27-9bea-bf515566a7fe None admin] RBAC: Adding query filter params (name=admin) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:49:44.009892 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bab6bc5-8aa1-4e27-9bea-bf515566a7fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:49:44.011239 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bab6bc5-8aa1-4e27-9bea-bf515566a7fe None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:49:44.018964 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 624/1264] 10.222.0.64 () {56 vars in 1099 bytes} [Tue Aug 2 09:49:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:49:44.024663 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8160ba52-aa46-4755-a2f1-32042f37de63 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:49:44.090263 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:49:44.123628 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8160ba52-aa46-4755-a2f1-32042f37de63 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:49:44.156821 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b36280-cdb4-40c6-9e4e-dcbd88a5fb74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:49:44.157450 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b36280-cdb4-40c6-9e4e-dcbd88a5fb74 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:44.157737 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b36280-cdb4-40c6-9e4e-dcbd88a5fb74 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:44.158013 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b36280-cdb4-40c6-9e4e-dcbd88a5fb74 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:44.168162 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b36280-cdb4-40c6-9e4e-dcbd88a5fb74 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:49:44.169645 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b36280-cdb4-40c6-9e4e-dcbd88a5fb74 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:49:44.185162 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-92b36280-cdb4-40c6-9e4e-dcbd88a5fb74 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:49:44.185162 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:49:44.185162 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:49:44.185162 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:49:44.185162 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:49:44.185162 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:49:44.185162 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:49:44.185162 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:49:44.185162 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:49:44.185162 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:49:44.185162 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:49:44.185162 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:49:44.185162 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 09:49:44.185162 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 09:49:44.185162 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 09:49:44.187148 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 09:49:44.187148 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:49:44.187148 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:49:44.187148 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:49:44.187148 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:49:44.187148 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:49:44.187148 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:49:44.187148 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:49:44.187148 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:49:44.187148 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:49:44.187148 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:49:44.187148 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:49:44.187148 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:49:44.187148 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:49:44.187148 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:49:44.187148 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:49:44.187148 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:49:44.189343 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 09:49:44.189343 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 09:49:44.189343 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:49:44.189343 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:49:44.189343 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 641/1265] 10.222.0.64 () {56 vars in 1108 bytes} [Tue Aug 2 09:49:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:49:44.192119 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bab6bc5-8aa1-4e27-9bea-bf515566a7fe None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:49:44.278246 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:49:44.297900 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bab6bc5-8aa1-4e27-9bea-bf515566a7fe None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:49:44.332897 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cf2256-e808-4dc9-ba17-0f4f1518259e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:49:44.333528 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cf2256-e808-4dc9-ba17-0f4f1518259e None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:44.333816 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cf2256-e808-4dc9-ba17-0f4f1518259e None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:44.334095 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cf2256-e808-4dc9-ba17-0f4f1518259e None admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:44.334802 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cf2256-e808-4dc9-ba17-0f4f1518259e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:49:44.335111 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cf2256-e808-4dc9-ba17-0f4f1518259e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:49:44.336513 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cf2256-e808-4dc9-ba17-0f4f1518259e None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:49:44.348807 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 625/1266] 10.222.0.64 () {56 vars in 1109 bytes} [Tue Aug 2 09:49:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:49:44.355036 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b36280-cdb4-40c6-9e4e-dcbd88a5fb74 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:49:44.460816 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b36280-cdb4-40c6-9e4e-dcbd88a5fb74 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:49:44.481363 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:49:44.495444 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe890499-9a71-4aa1-99b3-676353a654af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:49:44.496076 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe890499-9a71-4aa1-99b3-676353a654af None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:44.496391 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe890499-9a71-4aa1-99b3-676353a654af None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:44.496667 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe890499-9a71-4aa1-99b3-676353a654af None admin] PATH_INFO: `/v3/users/placement` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:44.505242 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe890499-9a71-4aa1-99b3-676353a654af None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:49:44.506696 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe890499-9a71-4aa1-99b3-676353a654af None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:49:44.518560 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-fe890499-9a71-4aa1-99b3-676353a654af None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 02 09:49:44.518560 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:49:44.518560 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:49:44.518560 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:49:44.518560 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:49:44.518560 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:49:44.518560 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:49:44.518560 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:49:44.518560 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:49:44.518560 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:49:44.518560 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:49:44.518560 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:49:44.518560 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 09:49:44.518560 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 09:49:44.518560 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 09:49:44.518560 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 09:49:44.520174 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:49:44.520174 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:49:44.520174 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 09:49:44.520174 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 09:49:44.520174 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 09:49:44.520174 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 09:49:44.520174 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:49:44.520174 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:49:44.520174 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:49:44.520174 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:49:44.520174 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:49:44.520174 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:49:44.520174 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:49:44.520174 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:49:44.520174 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:49:44.520174 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:49:44.520174 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:49:44.520174 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:49:44.521903 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:49:44.521903 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:49:44.521903 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 09:49:44.521903 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 09:49:44.521903 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 09:49:44.521903 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 09:49:44.521903 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 09:49:44.521903 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 09:49:44.521903 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 02 09:49:44.521903 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:49:44.521903 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 642/1267] 10.222.0.64 () {56 vars in 1108 bytes} [Tue Aug 2 09:49:44 2022] GET /identity/v3/users/placement => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 09:49:44.526806 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cf2256-e808-4dc9-ba17-0f4f1518259e None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:49:44.623775 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cf2256-e808-4dc9-ba17-0f4f1518259e None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:49:44.657280 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63f7be2-d4fd-4bfa-ba05-eee470815170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:49:44.657896 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63f7be2-d4fd-4bfa-ba05-eee470815170 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:44.658177 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63f7be2-d4fd-4bfa-ba05-eee470815170 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:44.658446 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63f7be2-d4fd-4bfa-ba05-eee470815170 None admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:44.659239 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63f7be2-d4fd-4bfa-ba05-eee470815170 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:49:44.659545 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63f7be2-d4fd-4bfa-ba05-eee470815170 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:49:44.660987 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63f7be2-d4fd-4bfa-ba05-eee470815170 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:49:44.669616 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:49:44.678509 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 626/1268] 10.222.0.64 () {56 vars in 1143 bytes} [Tue Aug 2 09:49:44 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:49:44.684744 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe890499-9a71-4aa1-99b3-676353a654af None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:49:44.798248 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe890499-9a71-4aa1-99b3-676353a654af None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:49:44.832125 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8c7d51-032c-4860-a05a-271c8fbe4030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:49:44.832791 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8c7d51-032c-4860-a05a-271c8fbe4030 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:44.833073 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8c7d51-032c-4860-a05a-271c8fbe4030 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:44.833348 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8c7d51-032c-4860-a05a-271c8fbe4030 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:44.843741 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8c7d51-032c-4860-a05a-271c8fbe4030 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:49:44.845503 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8c7d51-032c-4860-a05a-271c8fbe4030 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:49:44.859812 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-ce8c7d51-032c-4860-a05a-271c8fbe4030 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:49:44.859812 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:49:44.859812 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:49:44.859812 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:49:44.859812 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:49:44.859812 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:49:44.859812 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:49:44.859812 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:49:44.859812 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:49:44.859812 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:49:44.859812 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:49:44.859812 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:49:44.859812 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 09:49:44.859812 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 09:49:44.859812 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 09:49:44.861028 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 09:49:44.861028 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:49:44.861028 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:49:44.861028 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:49:44.861028 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:49:44.861028 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:49:44.861028 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:49:44.861028 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:49:44.861028 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:49:44.861028 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:49:44.861028 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:49:44.861028 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:49:44.861028 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:49:44.861028 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:49:44.861028 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:49:44.861028 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:49:44.861028 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:49:44.862240 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 09:49:44.862240 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 09:49:44.862240 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:49:44.862240 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:49:44.862240 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 643/1269] 10.222.0.64 () {56 vars in 1108 bytes} [Tue Aug 2 09:49:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:49:44.866726 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63f7be2-d4fd-4bfa-ba05-eee470815170 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:49:44.965890 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63f7be2-d4fd-4bfa-ba05-eee470815170 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:49:45.000197 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeed0de-b906-4ace-9c9e-1e3a4a0b6754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:49:45.000844 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeed0de-b906-4ace-9c9e-1e3a4a0b6754 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:45.001130 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeed0de-b906-4ace-9c9e-1e3a4a0b6754 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:45.001406 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeed0de-b906-4ace-9c9e-1e3a4a0b6754 None admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:45.002106 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdeed0de-b906-4ace-9c9e-1e3a4a0b6754 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:49:45.002412 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdeed0de-b906-4ace-9c9e-1e3a4a0b6754 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:49:45.003779 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdeed0de-b906-4ace-9c9e-1e3a4a0b6754 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:49:45.015739 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 627/1270] 10.222.0.64 () {56 vars in 1109 bytes} [Tue Aug 2 09:49:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:49:45.021447 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8c7d51-032c-4860-a05a-271c8fbe4030 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:49:45.053928 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:49:45.122904 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8c7d51-032c-4860-a05a-271c8fbe4030 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:49:45.157167 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e50f14-3b48-41b0-b4c6-c795a19cc61b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:49:45.157801 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e50f14-3b48-41b0-b4c6-c795a19cc61b None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:45.158080 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e50f14-3b48-41b0-b4c6-c795a19cc61b None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:45.158350 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e50f14-3b48-41b0-b4c6-c795a19cc61b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:45.165141 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e50f14-3b48-41b0-b4c6-c795a19cc61b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:49:45.166583 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e50f14-3b48-41b0-b4c6-c795a19cc61b None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:49:45.177979 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-a7e50f14-3b48-41b0-b4c6-c795a19cc61b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 09:49:45.177979 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:49:45.177979 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:49:45.177979 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:49:45.177979 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:49:45.177979 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:49:45.177979 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:49:45.177979 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:49:45.177979 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:49:45.177979 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:49:45.177979 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:49:45.177979 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:49:45.177979 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 09:49:45.177979 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 09:49:45.177979 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 09:49:45.179128 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 09:49:45.179128 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:49:45.179128 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:49:45.179128 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:49:45.179128 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:49:45.179128 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:49:45.179128 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:49:45.179128 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:49:45.179128 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:49:45.179128 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:49:45.179128 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:49:45.179128 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:49:45.179128 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:49:45.179128 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:49:45.179128 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:49:45.179128 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:49:45.179128 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:49:45.180110 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 09:49:45.180110 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 09:49:45.180110 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 09:49:45.180110 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 09:49:45.180110 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 09:49:45.180110 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 09:49:45.180110 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 09:49:45.180110 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:49:45.180110 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 644/1271] 10.222.0.64 () {56 vars in 1111 bytes} [Tue Aug 2 09:49:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:49:45.185041 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeed0de-b906-4ace-9c9e-1e3a4a0b6754 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:49:45.245209 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:49:45.285888 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeed0de-b906-4ace-9c9e-1e3a4a0b6754 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:49:45.318905 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc74e0b6-f254-4e6e-830a-55c76a7ff2f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:49:45.319534 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc74e0b6-f254-4e6e-830a-55c76a7ff2f9 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:45.319848 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc74e0b6-f254-4e6e-830a-55c76a7ff2f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:45.320119 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc74e0b6-f254-4e6e-830a-55c76a7ff2f9 None admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:45.320907 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc74e0b6-f254-4e6e-830a-55c76a7ff2f9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:49:45.321210 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc74e0b6-f254-4e6e-830a-55c76a7ff2f9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:49:45.322630 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc74e0b6-f254-4e6e-830a-55c76a7ff2f9 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:49:45.334737 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 628/1272] 10.222.0.64 () {56 vars in 1148 bytes} [Tue Aug 2 09:49:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:49:45.341250 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e50f14-3b48-41b0-b4c6-c795a19cc61b None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:49:45.444024 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:49:45.453522 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e50f14-3b48-41b0-b4c6-c795a19cc61b None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:49:45.486379 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1442d94d-3d49-4488-a277-a99bdef85503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:49:45.487041 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1442d94d-3d49-4488-a277-a99bdef85503 None admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:45.487333 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1442d94d-3d49-4488-a277-a99bdef85503 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:45.487612 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1442d94d-3d49-4488-a277-a99bdef85503 None admin] PATH_INFO: `/v3/projects/390bd7c4f91c4465b0c7ab0f9e436c4d/users/b770f073c86b4d88837a8b1f796a8b1d/roles/1000cf318fa14915b8091599eaead5de` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:45.489550 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1442d94d-3d49-4488-a277-a99bdef85503 None admin] RBAC: Authorizing `identity:create_grant(project_id=390bd7c4f91c4465b0c7ab0f9e436c4d, user_id=b770f073c86b4d88837a8b1f796a8b1d, role_id=1000cf318fa14915b8091599eaead5de)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:49:45.490993 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1442d94d-3d49-4488-a277-a99bdef85503 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:49:45.503974 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 645/1273] 10.222.0.64 () {58 vars in 1439 bytes} [Tue Aug 2 09:49:45 2022] PUT /identity/v3/projects/390bd7c4f91c4465b0c7ab0f9e436c4d/users/b770f073c86b4d88837a8b1f796a8b1d/roles/1000cf318fa14915b8091599eaead5de => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:49:47.494762 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb495e2b-1d90-4495-8c94-4eafb043fdb9 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:47.495301 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb495e2b-1d90-4495-8c94-4eafb043fdb9 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:47.495301 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb495e2b-1d90-4495-8c94-4eafb043fdb9 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:47.496692 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 629/1274] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:49:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:49:47.501950 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a99cd2-10ee-4c1c-8333-f3432ece1954 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:47.502378 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a99cd2-10ee-4c1c-8333-f3432ece1954 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:47.502514 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a99cd2-10ee-4c1c-8333-f3432ece1954 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:47.520753 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-d5a99cd2-10ee-4c1c-8333-f3432ece1954 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:49:47.521942 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-d5a99cd2-10ee-4c1c-8333-f3432ece1954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:49:47.585962 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 646/1275] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:49:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 09:49:47.609296 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb495e2b-1d90-4495-8c94-4eafb043fdb9 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:49:47.708511 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb495e2b-1d90-4495-8c94-4eafb043fdb9 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:49:47.725507 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:49:47.742779 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29d7e8d-90cc-440e-a753-1a8c5fb0a786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:49:47.743421 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29d7e8d-90cc-440e-a753-1a8c5fb0a786 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:47.743708 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29d7e8d-90cc-440e-a753-1a8c5fb0a786 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:47.743984 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29d7e8d-90cc-440e-a753-1a8c5fb0a786 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:47.753861 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29d7e8d-90cc-440e-a753-1a8c5fb0a786 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 09:49:47.759243 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-b29d7e8d-90cc-440e-a753-1a8c5fb0a786 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 09:49:47.759243 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:49:47.759243 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:49:47.759243 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:49:47.759243 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:49:47.759243 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:49:47.759243 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:49:47.759243 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:49:47.759243 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:49:47.759243 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:49:47.759243 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:49:47.759243 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:49:47.759243 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 09:49:47.759243 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 09:49:47.759243 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 09:49:47.759243 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 09:49:47.760654 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 09:49:47.760654 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 09:49:47.760654 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 09:49:47.760654 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 09:49:47.760654 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:49:47.760654 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:49:47.760654 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:49:47.760654 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:49:47.760654 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:49:47.760654 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:49:47.760654 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:49:47.760654 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:49:47.760654 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:49:47.760654 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:49:47.760654 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:49:47.760654 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:49:47.761922 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:49:47.761922 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:49:47.761922 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:49:47.761922 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:49:47.761922 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 09:49:47.761922 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 09:49:47.761922 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 09:49:47.761922 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 09:49:47.761922 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 09:49:47.761922 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 09:49:47.761922 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 09:49:47.761922 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:49:47.761922 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 630/1276] 10.222.0.64 () {56 vars in 1096 bytes} [Tue Aug 2 09:49:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:49:47.766555 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a99cd2-10ee-4c1c-8333-f3432ece1954 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:49:47.845294 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:49:47.863973 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a99cd2-10ee-4c1c-8333-f3432ece1954 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:49:47.900542 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c30045-383e-4f60-a3e2-493553777500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:49:47.901597 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c30045-383e-4f60-a3e2-493553777500 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:47.901883 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c30045-383e-4f60-a3e2-493553777500 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:47.902162 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c30045-383e-4f60-a3e2-493553777500 None admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:47.902879 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c30045-383e-4f60-a3e2-493553777500 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:49:47.903190 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c30045-383e-4f60-a3e2-493553777500 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:49:47.904590 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c30045-383e-4f60-a3e2-493553777500 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:49:47.912597 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 647/1277] 10.222.0.64 () {56 vars in 1099 bytes} [Tue Aug 2 09:49:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:49:47.918408 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29d7e8d-90cc-440e-a753-1a8c5fb0a786 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:49:48.016740 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29d7e8d-90cc-440e-a753-1a8c5fb0a786 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:49:48.049614 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d56408e-f432-4565-966e-087b9afcd2d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:49:48.050249 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d56408e-f432-4565-966e-087b9afcd2d0 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:48.050532 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d56408e-f432-4565-966e-087b9afcd2d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:48.050806 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d56408e-f432-4565-966e-087b9afcd2d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:48.061428 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d56408e-f432-4565-966e-087b9afcd2d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:49:48.062868 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d56408e-f432-4565-966e-087b9afcd2d0 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:49:48.077186 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-9d56408e-f432-4565-966e-087b9afcd2d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:49:48.077186 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:49:48.077186 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:49:48.077186 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:49:48.077186 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:49:48.077186 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:49:48.077186 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:49:48.077186 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:49:48.077186 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:49:48.077186 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:49:48.077186 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:49:48.077186 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:49:48.077186 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 09:49:48.077186 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 09:49:48.077186 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 09:49:48.078279 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 09:49:48.078279 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:49:48.078279 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:49:48.078279 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:49:48.078279 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:49:48.078279 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:49:48.078279 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:49:48.078279 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:49:48.078279 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:49:48.078279 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:49:48.078279 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:49:48.078279 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:49:48.078279 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:49:48.078279 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:49:48.078279 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:49:48.078279 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:49:48.078279 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:49:48.084267 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 09:49:48.084267 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 09:49:48.084267 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:49:48.084267 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:49:48.084267 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 631/1278] 10.222.0.64 () {56 vars in 1108 bytes} [Tue Aug 2 09:49:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:49:48.084687 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c30045-383e-4f60-a3e2-493553777500 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:49:48.184217 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c30045-383e-4f60-a3e2-493553777500 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:49:48.218329 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e4209e-9315-4af5-9bd5-adf58574ed5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:49:48.218939 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e4209e-9315-4af5-9bd5-adf58574ed5c None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:48.219221 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e4209e-9315-4af5-9bd5-adf58574ed5c None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:48.219495 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e4209e-9315-4af5-9bd5-adf58574ed5c None admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:48.220469 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e4209e-9315-4af5-9bd5-adf58574ed5c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:49:48.220775 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e4209e-9315-4af5-9bd5-adf58574ed5c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:49:48.222144 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e4209e-9315-4af5-9bd5-adf58574ed5c None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:49:48.233892 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 648/1279] 10.222.0.64 () {56 vars in 1109 bytes} [Tue Aug 2 09:49:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:49:48.239720 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d56408e-f432-4565-966e-087b9afcd2d0 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:49:48.270587 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:49:48.340213 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d56408e-f432-4565-966e-087b9afcd2d0 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:49:48.374999 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cee1e3-2d4f-4d19-a178-d47752cffebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:49:48.375674 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cee1e3-2d4f-4d19-a178-d47752cffebc None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:48.375981 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cee1e3-2d4f-4d19-a178-d47752cffebc None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:48.376352 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cee1e3-2d4f-4d19-a178-d47752cffebc None admin] PATH_INFO: `/v3/users/placement` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:48.386113 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cee1e3-2d4f-4d19-a178-d47752cffebc None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:49:48.387782 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cee1e3-2d4f-4d19-a178-d47752cffebc None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:49:48.400848 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-91cee1e3-2d4f-4d19-a178-d47752cffebc None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 02 09:49:48.400848 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:49:48.400848 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:49:48.400848 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:49:48.400848 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:49:48.400848 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:49:48.400848 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:49:48.400848 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:49:48.400848 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:49:48.400848 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:49:48.400848 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:49:48.400848 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:49:48.400848 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 09:49:48.400848 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 09:49:48.400848 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 09:49:48.400848 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 09:49:48.402267 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:49:48.402267 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:49:48.402267 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 09:49:48.402267 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 09:49:48.402267 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 09:49:48.402267 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 09:49:48.402267 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:49:48.402267 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:49:48.402267 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:49:48.402267 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:49:48.402267 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:49:48.402267 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:49:48.402267 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:49:48.402267 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:49:48.402267 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:49:48.402267 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:49:48.402267 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:49:48.402267 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:49:48.403469 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:49:48.403469 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:49:48.403469 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 09:49:48.403469 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 09:49:48.403469 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 09:49:48.403469 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 09:49:48.403469 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 09:49:48.403469 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 09:49:48.403469 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 02 09:49:48.403469 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:49:48.403469 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 632/1280] 10.222.0.64 () {56 vars in 1108 bytes} [Tue Aug 2 09:49:48 2022] GET /identity/v3/users/placement => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:49:48.408767 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e4209e-9315-4af5-9bd5-adf58574ed5c None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:49:48.413160 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:49:48.513553 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e4209e-9315-4af5-9bd5-adf58574ed5c None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:49:48.546531 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676d4d2d-f580-4df1-88f2-bbf3688f97b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:49:48.547079 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676d4d2d-f580-4df1-88f2-bbf3688f97b4 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:48.547364 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676d4d2d-f580-4df1-88f2-bbf3688f97b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:48.547643 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676d4d2d-f580-4df1-88f2-bbf3688f97b4 None admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:48.548467 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676d4d2d-f580-4df1-88f2-bbf3688f97b4 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:49:48.548781 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676d4d2d-f580-4df1-88f2-bbf3688f97b4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:49:48.550215 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676d4d2d-f580-4df1-88f2-bbf3688f97b4 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:49:48.567509 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 649/1281] 10.222.0.64 () {56 vars in 1143 bytes} [Tue Aug 2 09:49:48 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:49:48.573906 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cee1e3-2d4f-4d19-a178-d47752cffebc None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:49:48.661649 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:49:48.671093 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cee1e3-2d4f-4d19-a178-d47752cffebc None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:49:48.703837 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1481de4-05bb-4975-9c8d-9c93647d7fa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:49:48.704476 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1481de4-05bb-4975-9c8d-9c93647d7fa6 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:48.704757 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1481de4-05bb-4975-9c8d-9c93647d7fa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:48.705029 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1481de4-05bb-4975-9c8d-9c93647d7fa6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:48.769197 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1481de4-05bb-4975-9c8d-9c93647d7fa6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:49:48.770628 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1481de4-05bb-4975-9c8d-9c93647d7fa6 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:49:48.855089 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:49:48.865384 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 633/1282] 10.222.0.64 () {58 vars in 1282 bytes} [Tue Aug 2 09:49:48 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 02 09:49:48.871870 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676d4d2d-f580-4df1-88f2-bbf3688f97b4 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:49:48.933548 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:49:48.978567 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676d4d2d-f580-4df1-88f2-bbf3688f97b4 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:49:49.012098 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3afd20-700e-4ca6-9ecb-3bed274b8563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:49:49.012760 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3afd20-700e-4ca6-9ecb-3bed274b8563 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:49.013040 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3afd20-700e-4ca6-9ecb-3bed274b8563 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:49.013309 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3afd20-700e-4ca6-9ecb-3bed274b8563 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:49.023065 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3afd20-700e-4ca6-9ecb-3bed274b8563 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:49:49.024537 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3afd20-700e-4ca6-9ecb-3bed274b8563 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:49:49.039653 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-2b3afd20-700e-4ca6-9ecb-3bed274b8563 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:49:49.039653 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:49:49.039653 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:49:49.039653 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:49:49.039653 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:49:49.039653 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:49:49.039653 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:49:49.039653 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:49:49.039653 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:49:49.039653 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:49:49.039653 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:49:49.039653 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:49:49.039653 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 09:49:49.039653 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 09:49:49.039653 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 09:49:49.040814 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 09:49:49.040814 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:49:49.040814 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:49:49.040814 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:49:49.040814 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:49:49.040814 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:49:49.040814 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:49:49.040814 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:49:49.040814 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:49:49.040814 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:49:49.040814 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:49:49.040814 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:49:49.040814 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:49:49.040814 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:49:49.040814 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:49:49.040814 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:49:49.040814 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:49:49.041941 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 09:49:49.041941 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 09:49:49.041941 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 09:49:49.041941 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:49:49.041941 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 650/1283] 10.222.0.64 () {56 vars in 1108 bytes} [Tue Aug 2 09:49:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:49:49.048330 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1481de4-05bb-4975-9c8d-9c93647d7fa6 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:49:49.148354 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1481de4-05bb-4975-9c8d-9c93647d7fa6 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:49:49.182124 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c94d34-7749-4111-858d-706878ccdada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:49:49.182744 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c94d34-7749-4111-858d-706878ccdada None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:49.183025 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c94d34-7749-4111-858d-706878ccdada None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:49.183304 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c94d34-7749-4111-858d-706878ccdada None admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:49.183994 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c94d34-7749-4111-858d-706878ccdada None admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:49:49.184324 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c94d34-7749-4111-858d-706878ccdada None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:49:49.185686 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c94d34-7749-4111-858d-706878ccdada None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:49:49.197420 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 634/1284] 10.222.0.64 () {56 vars in 1109 bytes} [Tue Aug 2 09:49:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:49:49.203062 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3afd20-700e-4ca6-9ecb-3bed274b8563 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:49:49.302411 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3afd20-700e-4ca6-9ecb-3bed274b8563 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:49:49.315736 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:49:49.335236 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b6c8be-ec5d-473a-be3b-99eae8ee7636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:49:49.335868 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b6c8be-ec5d-473a-be3b-99eae8ee7636 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:49.336686 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b6c8be-ec5d-473a-be3b-99eae8ee7636 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:49.336964 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b6c8be-ec5d-473a-be3b-99eae8ee7636 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:49.343439 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b6c8be-ec5d-473a-be3b-99eae8ee7636 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:49:49.345185 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b6c8be-ec5d-473a-be3b-99eae8ee7636 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:49:49.357669 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-83b6c8be-ec5d-473a-be3b-99eae8ee7636 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 09:49:49.357669 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:49:49.357669 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:49:49.357669 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:49:49.357669 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:49:49.357669 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:49:49.357669 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:49:49.357669 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:49:49.357669 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:49:49.357669 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:49:49.357669 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:49:49.357669 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:49:49.357669 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 09:49:49.357669 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 09:49:49.357669 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 09:49:49.358737 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 09:49:49.358737 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:49:49.358737 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:49:49.358737 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:49:49.358737 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:49:49.358737 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:49:49.358737 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:49:49.358737 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:49:49.358737 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Aug 02 09:49:49.358737 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:49:49.358737 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:49:49.358737 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:49:49.358737 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:49:49.358737 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:49:49.358737 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:49:49.358737 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:49:49.358737 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:49:49.359829 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 09:49:49.359829 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 09:49:49.359829 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 09:49:49.359829 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 09:49:49.359829 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 09:49:49.359829 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 09:49:49.359829 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 09:49:49.359829 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Aug 02 09:49:49.359829 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 651/1285] 10.222.0.64 () {56 vars in 1111 bytes} [Tue Aug 2 09:49:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:49:49.364696 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c94d34-7749-4111-858d-706878ccdada None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:49:49.422648 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:49:49.473792 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c94d34-7749-4111-858d-706878ccdada None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:49:49.507267 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6af42c0-4101-43e3-b082-73a646891332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:49:49.508405 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6af42c0-4101-43e3-b082-73a646891332 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:49.508887 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6af42c0-4101-43e3-b082-73a646891332 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:49.509166 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6af42c0-4101-43e3-b082-73a646891332 None admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:49.509953 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6af42c0-4101-43e3-b082-73a646891332 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:49:49.510278 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6af42c0-4101-43e3-b082-73a646891332 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:49:49.511708 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6af42c0-4101-43e3-b082-73a646891332 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:49:49.526255 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 635/1286] 10.222.0.64 () {56 vars in 1148 bytes} [Tue Aug 2 09:49:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:49:49.532566 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b6c8be-ec5d-473a-be3b-99eae8ee7636 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:49:49.639740 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b6c8be-ec5d-473a-be3b-99eae8ee7636 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:49:49.675943 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9104f658-7d19-48a9-825f-a893497006ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:49:49.676500 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9104f658-7d19-48a9-825f-a893497006ac None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:49.676786 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9104f658-7d19-48a9-825f-a893497006ac None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:49.677060 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9104f658-7d19-48a9-825f-a893497006ac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:49.677853 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9104f658-7d19-48a9-825f-a893497006ac None admin] RBAC: Adding query filter params (role.id=1000cf318fa14915b8091599eaead5de, scope.project.id=390bd7c4f91c4465b0c7ab0f9e436c4d, user.id=b770f073c86b4d88837a8b1f796a8b1d) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:49:49.678163 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9104f658-7d19-48a9-825f-a893497006ac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:49:49.679613 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9104f658-7d19-48a9-825f-a893497006ac None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:49:49.686150 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 652/1287] 10.222.0.64 () {56 vars in 1374 bytes} [Tue Aug 2 09:49:49 2022] GET /identity/v3/role_assignments?user.id=b770f073c86b4d88837a8b1f796a8b1d&scope.project.id=390bd7c4f91c4465b0c7ab0f9e436c4d&role.id=1000cf318fa14915b8091599eaead5de => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:49:51.704738 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203d942f-6cdf-47e0-81ad-3af4ad8cfeac None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:51.705297 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203d942f-6cdf-47e0-81ad-3af4ad8cfeac None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:51.705297 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203d942f-6cdf-47e0-81ad-3af4ad8cfeac None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:51.706645 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 636/1288] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:49:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:49:51.712323 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ec281b-4466-465a-a505-12bf3a327fbb None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:51.712702 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ec281b-4466-465a-a505-12bf3a327fbb None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:51.712881 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ec281b-4466-465a-a505-12bf3a327fbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:51.731484 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-a2ec281b-4466-465a-a505-12bf3a327fbb None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:49:51.732732 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-a2ec281b-4466-465a-a505-12bf3a327fbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:49:51.761529 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:49:51.798029 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 653/1289] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:49:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 09:49:51.819664 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203d942f-6cdf-47e0-81ad-3af4ad8cfeac None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:49:51.921703 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203d942f-6cdf-47e0-81ad-3af4ad8cfeac None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:49:51.932112 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:49:51.955545 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf1b2ad-a42f-4363-af04-37a9917ea511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:49:51.956111 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf1b2ad-a42f-4363-af04-37a9917ea511 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:51.956423 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf1b2ad-a42f-4363-af04-37a9917ea511 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:51.956701 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf1b2ad-a42f-4363-af04-37a9917ea511 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:51.957324 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf1b2ad-a42f-4363-af04-37a9917ea511 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:49:51.958679 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf1b2ad-a42f-4363-af04-37a9917ea511 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:49:51.967983 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-bcf1b2ad-a42f-4363-af04-37a9917ea511 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 02 09:49:51.967983 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:49:51.967983 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 09:49:51.967983 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 09:49:51.967983 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 09:49:51.967983 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 09:49:51.967983 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 09:49:51.967983 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 09:49:51.967983 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 02 09:49:51.967983 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:49:51.967983 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 09:49:51.967983 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:49:51.967983 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:49:51.967983 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:49:51.967983 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:49:51.967983 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:49:51.967983 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:49:51.969345 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:49:51.969345 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:49:51.969345 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:49:51.969345 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:49:51.969345 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:49:51.969345 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:49:51.969345 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 09:49:51.969345 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 09:49:51.969345 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 09:49:51.969345 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 09:49:51.969345 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:49:51.969345 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:49:51.969345 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:49:51.969345 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:49:51.969345 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:49:51.969345 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:49:51.970710 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:49:51.970710 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:49:51.970710 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:49:51.970710 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:49:51.970710 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:49:51.970710 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:49:51.970710 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:49:51.970710 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:49:51.970710 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:49:51.970710 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:49:51.970710 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 09:49:51.970710 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 09:49:51.970710 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 02 09:49:51.970710 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:49:51.970710 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 637/1290] 10.222.0.64 () {56 vars in 1117 bytes} [Tue Aug 2 09:49:51 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:49:51.975559 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ec281b-4466-465a-a505-12bf3a327fbb None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:49:52.072509 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ec281b-4466-465a-a505-12bf3a327fbb None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:49:52.106281 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deeb1735-e5c2-4f07-a838-58e826455a40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:49:52.106897 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeb1735-e5c2-4f07-a838-58e826455a40 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:52.107182 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeb1735-e5c2-4f07-a838-58e826455a40 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:52.107458 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeb1735-e5c2-4f07-a838-58e826455a40 None admin] PATH_INFO: `/v3/services` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:52.108138 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deeb1735-e5c2-4f07-a838-58e826455a40 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:49:52.108480 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deeb1735-e5c2-4f07-a838-58e826455a40 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:49:52.109844 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deeb1735-e5c2-4f07-a838-58e826455a40 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:49:52.114622 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 654/1291] 10.222.0.64 () {56 vars in 1116 bytes} [Tue Aug 2 09:49:51 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:49:52.120583 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf1b2ad-a42f-4363-af04-37a9917ea511 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:49:52.218657 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf1b2ad-a42f-4363-af04-37a9917ea511 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:49:52.252537 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5094e7b-b060-482f-9d2c-ea6104e30a74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:49:52.253175 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5094e7b-b060-482f-9d2c-ea6104e30a74 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:52.253460 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5094e7b-b060-482f-9d2c-ea6104e30a74 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:52.253735 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5094e7b-b060-482f-9d2c-ea6104e30a74 None admin] PATH_INFO: `/v3/services` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:52.254435 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5094e7b-b060-482f-9d2c-ea6104e30a74 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:49:52.254740 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5094e7b-b060-482f-9d2c-ea6104e30a74 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:49:52.256133 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5094e7b-b060-482f-9d2c-ea6104e30a74 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:49:52.261858 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 638/1292] 10.222.0.64 () {56 vars in 1116 bytes} [Tue Aug 2 09:49:52 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:49:54.246215 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15e3ef3-efa2-48e2-8184-32a66ba2fab1 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:54.246769 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15e3ef3-efa2-48e2-8184-32a66ba2fab1 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:54.246769 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15e3ef3-efa2-48e2-8184-32a66ba2fab1 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:54.248026 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 655/1293] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:49:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:49:54.253234 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a14929-2efe-42b5-93a7-bdae5ce3f933 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:54.253657 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a14929-2efe-42b5-93a7-bdae5ce3f933 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:54.253803 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a14929-2efe-42b5-93a7-bdae5ce3f933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:54.271761 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-02a14929-2efe-42b5-93a7-bdae5ce3f933 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:49:54.272978 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-02a14929-2efe-42b5-93a7-bdae5ce3f933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:49:54.301905 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:49:54.343335 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 639/1294] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:49:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 02 09:49:54.366611 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15e3ef3-efa2-48e2-8184-32a66ba2fab1 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:49:54.405382 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:49:54.491438 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15e3ef3-efa2-48e2-8184-32a66ba2fab1 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:49:54.525033 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd782c5-2958-4c4b-a388-5eff198e2cc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:49:54.525563 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd782c5-2958-4c4b-a388-5eff198e2cc2 None admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:54.525849 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd782c5-2958-4c4b-a388-5eff198e2cc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:54.526124 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd782c5-2958-4c4b-a388-5eff198e2cc2 None admin] PATH_INFO: `/v3/services` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:54.526816 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd782c5-2958-4c4b-a388-5eff198e2cc2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:49:54.528193 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd782c5-2958-4c4b-a388-5eff198e2cc2 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:49:54.543225 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 656/1295] 10.222.0.64 () {60 vars in 1141 bytes} [Tue Aug 2 09:49:54 2022] POST /identity/v3/services => generated 246 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:49:56.526306 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0e282f-8cfc-4d83-8fba-7ec86b55f1e7 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:56.526843 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0e282f-8cfc-4d83-8fba-7ec86b55f1e7 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:56.526843 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0e282f-8cfc-4d83-8fba-7ec86b55f1e7 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:56.528189 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 640/1296] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:49:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:49:56.533183 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df82738-be62-410c-8b4b-87d135874b32 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:56.533577 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df82738-be62-410c-8b4b-87d135874b32 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:56.533747 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df82738-be62-410c-8b4b-87d135874b32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:56.551392 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-9df82738-be62-410c-8b4b-87d135874b32 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:49:56.552612 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-9df82738-be62-410c-8b4b-87d135874b32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:49:56.624266 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 657/1297] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:49:56 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 09:49:56.647541 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0e282f-8cfc-4d83-8fba-7ec86b55f1e7 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:49:56.763831 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0e282f-8cfc-4d83-8fba-7ec86b55f1e7 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:49:56.800106 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f55b59-c888-4b2a-b70d-00c61c5d3ca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:49:56.800692 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f55b59-c888-4b2a-b70d-00c61c5d3ca4 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:56.800978 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f55b59-c888-4b2a-b70d-00c61c5d3ca4 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:56.801263 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f55b59-c888-4b2a-b70d-00c61c5d3ca4 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:56.801911 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f55b59-c888-4b2a-b70d-00c61c5d3ca4 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:49:56.803298 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f55b59-c888-4b2a-b70d-00c61c5d3ca4 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:49:56.813237 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-37f55b59-c888-4b2a-b70d-00c61c5d3ca4 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 02 09:49:56.813237 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:49:56.813237 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 09:49:56.813237 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 09:49:56.813237 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 09:49:56.813237 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 09:49:56.813237 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 09:49:56.813237 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 09:49:56.813237 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 02 09:49:56.813237 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:49:56.813237 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 09:49:56.813237 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:49:56.813237 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:49:56.813237 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:49:56.813237 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:49:56.813237 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:49:56.813237 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:49:56.814509 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:49:56.814509 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:49:56.814509 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:49:56.814509 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:49:56.814509 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:49:56.814509 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:49:56.814509 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 09:49:56.814509 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 09:49:56.814509 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 09:49:56.814509 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 09:49:56.814509 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:49:56.814509 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:49:56.814509 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:49:56.814509 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:49:56.814509 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:49:56.814509 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:49:56.815805 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:49:56.815805 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:49:56.815805 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:49:56.815805 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:49:56.815805 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:49:56.815805 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:49:56.815805 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:49:56.815805 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:49:56.815805 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:49:56.815805 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:49:56.815805 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 09:49:56.815805 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 09:49:56.815805 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 02 09:49:56.815805 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:49:56.815805 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 641/1298] 10.222.0.64 () {56 vars in 1117 bytes} [Tue Aug 2 09:49:56 2022] GET /identity/v3/services/placement => generated 90 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:49:56.821051 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df82738-be62-410c-8b4b-87d135874b32 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:49:56.839563 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:49:56.929929 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df82738-be62-410c-8b4b-87d135874b32 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:49:56.964333 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61dc117-769f-4677-9343-af375ac03ad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:49:56.964950 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61dc117-769f-4677-9343-af375ac03ad6 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:56.965236 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61dc117-769f-4677-9343-af375ac03ad6 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:56.965513 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61dc117-769f-4677-9343-af375ac03ad6 None admin] PATH_INFO: `/v3/services` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:56.966192 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61dc117-769f-4677-9343-af375ac03ad6 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:49:56.966498 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61dc117-769f-4677-9343-af375ac03ad6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:49:56.967869 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61dc117-769f-4677-9343-af375ac03ad6 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:49:56.973114 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 658/1299] 10.222.0.64 () {56 vars in 1116 bytes} [Tue Aug 2 09:49:56 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:49:56.979713 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f55b59-c888-4b2a-b70d-00c61c5d3ca4 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:49:57.010530 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:49:57.078875 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f55b59-c888-4b2a-b70d-00c61c5d3ca4 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:49:57.112350 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20224623-59af-4d09-8733-49f15411f467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:49:57.112974 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20224623-59af-4d09-8733-49f15411f467 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:57.113253 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20224623-59af-4d09-8733-49f15411f467 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:57.113533 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20224623-59af-4d09-8733-49f15411f467 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:57.114289 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20224623-59af-4d09-8733-49f15411f467 None admin] RBAC: Adding query filter params (interface=public, service_id=e1d40f8beb17444f915e9420c42cfd9c, region_id=RegionOne) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:49:57.114599 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20224623-59af-4d09-8733-49f15411f467 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:49:57.115967 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20224623-59af-4d09-8733-49f15411f467 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:49:57.121278 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 642/1300] 10.222.0.64 () {56 vars in 1251 bytes} [Tue Aug 2 09:49:56 2022] GET /identity/v3/endpoints?service_id=e1d40f8beb17444f915e9420c42cfd9c&interface=public®ion_id=RegionOne => generated 194 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:49:57.127204 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61dc117-769f-4677-9343-af375ac03ad6 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:49:57.224703 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:49:57.242332 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61dc117-769f-4677-9343-af375ac03ad6 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:49:57.275914 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f926255c-c678-4a4b-9858-c35ecdf75b4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:49:57.276552 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f926255c-c678-4a4b-9858-c35ecdf75b4c None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:57.276837 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f926255c-c678-4a4b-9858-c35ecdf75b4c None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:57.277109 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f926255c-c678-4a4b-9858-c35ecdf75b4c None admin] PATH_INFO: `/v3/services` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:57.277753 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f926255c-c678-4a4b-9858-c35ecdf75b4c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:49:57.279123 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f926255c-c678-4a4b-9858-c35ecdf75b4c None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:49:57.285742 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 659/1301] 10.222.0.64 () {56 vars in 1087 bytes} [Tue Aug 2 09:49:57 2022] GET /identity/v3/services => generated 2191 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:49:59.294875 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097a33ad-0ccb-4601-846f-0ea30210dd1c None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:59.295388 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097a33ad-0ccb-4601-846f-0ea30210dd1c None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:59.295388 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097a33ad-0ccb-4601-846f-0ea30210dd1c None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:59.296739 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 643/1302] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:49:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:49:59.302078 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ca34d2-a208-4e16-a8fc-6d6d3343765b None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:59.302527 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ca34d2-a208-4e16-a8fc-6d6d3343765b None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:59.302640 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ca34d2-a208-4e16-a8fc-6d6d3343765b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:59.321131 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-48ca34d2-a208-4e16-a8fc-6d6d3343765b None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:49:59.322323 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-48ca34d2-a208-4e16-a8fc-6d6d3343765b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:49:59.390834 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 660/1303] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:49:59 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 09:49:59.413145 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097a33ad-0ccb-4601-846f-0ea30210dd1c None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:49:59.506657 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:49:59.513573 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097a33ad-0ccb-4601-846f-0ea30210dd1c None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:49:59.546043 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b8eca8-dad2-4a97-bfb6-c3d612b9e494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:49:59.546615 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b8eca8-dad2-4a97-bfb6-c3d612b9e494 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:59.546901 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b8eca8-dad2-4a97-bfb6-c3d612b9e494 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:59.547177 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b8eca8-dad2-4a97-bfb6-c3d612b9e494 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:59.547801 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b8eca8-dad2-4a97-bfb6-c3d612b9e494 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:49:59.549191 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b8eca8-dad2-4a97-bfb6-c3d612b9e494 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:49:59.558421 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-e2b8eca8-dad2-4a97-bfb6-c3d612b9e494 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 02 09:49:59.558421 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:49:59.558421 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 09:49:59.558421 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 09:49:59.558421 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 09:49:59.558421 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 09:49:59.558421 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 09:49:59.558421 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 09:49:59.558421 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 02 09:49:59.558421 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:49:59.558421 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 09:49:59.558421 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:49:59.558421 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 09:49:59.558421 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 09:49:59.558421 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 09:49:59.558421 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 09:49:59.558421 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 09:49:59.559598 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 09:49:59.559598 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 09:49:59.559598 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 09:49:59.559598 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 09:49:59.559598 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 09:49:59.559598 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 09:49:59.559598 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 09:49:59.559598 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 09:49:59.559598 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 09:49:59.559598 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 09:49:59.559598 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 09:49:59.559598 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 09:49:59.559598 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 09:49:59.559598 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 09:49:59.559598 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 09:49:59.559598 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 09:49:59.560534 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 09:49:59.560534 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Aug 02 09:49:59.560534 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 09:49:59.560534 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Aug 02 09:49:59.560534 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 09:49:59.560534 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 09:49:59.560534 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 09:49:59.560534 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Aug 02 09:49:59.560534 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 09:49:59.560534 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Aug 02 09:49:59.560534 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 09:49:59.560534 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 09:49:59.560534 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 02 09:49:59.560534 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Aug 02 09:49:59.560534 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 644/1304] 10.222.0.64 () {56 vars in 1117 bytes} [Tue Aug 2 09:49:59 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 09:49:59.565793 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ca34d2-a208-4e16-a8fc-6d6d3343765b None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:49:59.639992 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:49:59.662971 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ca34d2-a208-4e16-a8fc-6d6d3343765b None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:49:59.696381 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef39b5a7-2c48-4ae6-b30c-642c17106c72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:49:59.696990 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef39b5a7-2c48-4ae6-b30c-642c17106c72 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:59.697272 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef39b5a7-2c48-4ae6-b30c-642c17106c72 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:59.697541 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef39b5a7-2c48-4ae6-b30c-642c17106c72 None admin] PATH_INFO: `/v3/services` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:59.698208 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef39b5a7-2c48-4ae6-b30c-642c17106c72 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:49:59.698507 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef39b5a7-2c48-4ae6-b30c-642c17106c72 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:49:59.699864 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef39b5a7-2c48-4ae6-b30c-642c17106c72 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:49:59.704869 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 661/1305] 10.222.0.64 () {56 vars in 1116 bytes} [Tue Aug 2 09:49:59 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:49:59.711256 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b8eca8-dad2-4a97-bfb6-c3d612b9e494 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:49:59.810291 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b8eca8-dad2-4a97-bfb6-c3d612b9e494 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:49:59.845886 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a062569-ee74-453a-8c3b-4f70735e5c95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:49:59.846513 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a062569-ee74-453a-8c3b-4f70735e5c95 None admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:49:59.846805 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a062569-ee74-453a-8c3b-4f70735e5c95 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:49:59.847081 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a062569-ee74-453a-8c3b-4f70735e5c95 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:49:59.847806 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a062569-ee74-453a-8c3b-4f70735e5c95 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:49:59.849203 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a062569-ee74-453a-8c3b-4f70735e5c95 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:49:59.869002 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 645/1306] 10.222.0.64 () {60 vars in 1144 bytes} [Tue Aug 2 09:49:59 2022] POST /identity/v3/endpoints => generated 331 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 09:50:09.702049 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=30 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:50:09.864879 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:51:03.161939 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2d3bf9-4a0a-4110-92b4-e7314b4bed29 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:51:03.162485 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2d3bf9-4a0a-4110-92b4-e7314b4bed29 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:51:03.162485 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2d3bf9-4a0a-4110-92b4-e7314b4bed29 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:51:03.168857 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca9278d-067c-4a13-8989-9637bf2aae1c None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:51:03.169297 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca9278d-067c-4a13-8989-9637bf2aae1c None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:51:03.169523 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca9278d-067c-4a13-8989-9637bf2aae1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:51:03.174192 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 662/1307] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:51:03 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:51:03.250071 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-1ca9278d-067c-4a13-8989-9637bf2aae1c None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:51:03.251561 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-1ca9278d-067c-4a13-8989-9637bf2aae1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:51:03.397156 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 646/1308] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:51:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:51:03.447957 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d229b57b-c709-421f-a54b-91f9472eca3d None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:51:03.448647 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d229b57b-c709-421f-a54b-91f9472eca3d None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:51:03.449240 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d229b57b-c709-421f-a54b-91f9472eca3d None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:51:03.450878 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 663/1309] 10.222.0.64 () {54 vars in 965 bytes} [Tue Aug 2 09:51:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:51:03.462869 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61896c48-ab2f-4462-8a05-e6f43488ae84 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:51:03.463209 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61896c48-ab2f-4462-8a05-e6f43488ae84 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:51:03.463675 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61896c48-ab2f-4462-8a05-e6f43488ae84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:51:03.593174 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-61896c48-ab2f-4462-8a05-e6f43488ae84 None None] MFA Rules not processed for user `ec31b188970947efb2ed1d4d9a31cadf`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:51:03.594572 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-61896c48-ab2f-4462-8a05-e6f43488ae84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:51:03.618322 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:51:03.676994 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 647/1310] 10.222.0.64 () {58 vars in 1062 bytes} [Tue Aug 2 09:51:03 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:51:03.685766 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d229b57b-c709-421f-a54b-91f9472eca3d None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:51:03.687766 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-d229b57b-c709-421f-a54b-91f9472eca3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:51:03.811532 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d229b57b-c709-421f-a54b-91f9472eca3d None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:51:03.908699 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ecd2f3-217c-43a0-ba8d-92c75eda8e62 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:51:03.909338 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ecd2f3-217c-43a0-ba8d-92c75eda8e62 service neutron] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:51:03.909620 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ecd2f3-217c-43a0-ba8d-92c75eda8e62 service neutron] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:51:03.909901 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ecd2f3-217c-43a0-ba8d-92c75eda8e62 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:51:03.911744 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-31ecd2f3-217c-43a0-ba8d-92c75eda8e62 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:51:03.978394 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:51:04.038423 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ecd2f3-217c-43a0-ba8d-92c75eda8e62 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:51:04.039875 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ecd2f3-217c-43a0-ba8d-92c75eda8e62 service neutron] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:51:04.059146 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 664/1311] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 09:51:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:51:06.344700 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd2e869-29fb-4c19-9306-79ffe2f66fad None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:51:06.345253 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd2e869-29fb-4c19-9306-79ffe2f66fad None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:51:06.345253 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd2e869-29fb-4c19-9306-79ffe2f66fad None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:51:06.346875 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 648/1312] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:51:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:51:06.352610 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4911713-a58b-4731-9bbf-991fdda11dab None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:51:06.353119 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4911713-a58b-4731-9bbf-991fdda11dab None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:51:06.353237 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4911713-a58b-4731-9bbf-991fdda11dab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:51:06.439941 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-d4911713-a58b-4731-9bbf-991fdda11dab None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:51:06.441107 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-d4911713-a58b-4731-9bbf-991fdda11dab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:51:06.552150 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 665/1313] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:51:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:51:06.589924 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd2e869-29fb-4c19-9306-79ffe2f66fad None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:51:06.591574 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-cbd2e869-29fb-4c19-9306-79ffe2f66fad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:51:06.702510 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd2e869-29fb-4c19-9306-79ffe2f66fad None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:51:06.775836 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:51:06.778560 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138d5f28-b29f-4fae-89c2-5ce4f7066dbb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:51:06.779467 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138d5f28-b29f-4fae-89c2-5ce4f7066dbb service neutron] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:51:06.779975 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138d5f28-b29f-4fae-89c2-5ce4f7066dbb service neutron] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:51:06.780526 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138d5f28-b29f-4fae-89c2-5ce4f7066dbb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:51:06.782208 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-138d5f28-b29f-4fae-89c2-5ce4f7066dbb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:51:06.892600 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138d5f28-b29f-4fae-89c2-5ce4f7066dbb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:51:06.894042 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138d5f28-b29f-4fae-89c2-5ce4f7066dbb service neutron] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:51:06.912281 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 649/1314] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 09:51:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:51:09.245683 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8788738-1411-4f04-b693-7149ffb935a3 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:51:09.246210 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8788738-1411-4f04-b693-7149ffb935a3 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:51:09.246210 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8788738-1411-4f04-b693-7149ffb935a3 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:51:09.247502 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 666/1315] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:51:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:51:09.252613 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dcf798-b1c9-4d3c-a925-0a74da755733 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:51:09.253077 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dcf798-b1c9-4d3c-a925-0a74da755733 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:51:09.253188 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dcf798-b1c9-4d3c-a925-0a74da755733 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:51:09.317363 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-41dcf798-b1c9-4d3c-a925-0a74da755733 None None] MFA Rules not processed for user `6b35b3486c4b4db6a107718cf2807616`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:51:09.318545 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-41dcf798-b1c9-4d3c-a925-0a74da755733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:51:09.370389 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:51:09.416615 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 650/1316] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:51:09 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:51:09.468746 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f236b2-fbb4-47f3-baf1-dad1f26e671f None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:51:09.469040 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f236b2-fbb4-47f3-baf1-dad1f26e671f None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:51:09.469324 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f236b2-fbb4-47f3-baf1-dad1f26e671f None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:51:09.470683 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 667/1317] 10.222.0.64 () {54 vars in 965 bytes} [Tue Aug 2 09:51:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:51:09.477072 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a51e548-9b18-4775-85f9-f38feb953307 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:51:09.477358 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a51e548-9b18-4775-85f9-f38feb953307 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:51:09.477632 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a51e548-9b18-4775-85f9-f38feb953307 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:51:09.553552 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-2a51e548-9b18-4775-85f9-f38feb953307 None None] MFA Rules not processed for user `ec31b188970947efb2ed1d4d9a31cadf`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:51:09.554668 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-2a51e548-9b18-4775-85f9-f38feb953307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:51:09.628427 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 651/1318] 10.222.0.64 () {58 vars in 1062 bytes} [Tue Aug 2 09:51:09 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:51:09.637355 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f236b2-fbb4-47f3-baf1-dad1f26e671f None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:51:09.638871 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-02f236b2-fbb4-47f3-baf1-dad1f26e671f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:51:09.649442 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:51:09.746129 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f236b2-fbb4-47f3-baf1-dad1f26e671f None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:51:09.820948 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e706d687-a35d-424e-962d-318e22fe0667 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:51:09.820948 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e706d687-a35d-424e-962d-318e22fe0667 service neutron] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:51:09.820948 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e706d687-a35d-424e-962d-318e22fe0667 service neutron] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:51:09.821387 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e706d687-a35d-424e-962d-318e22fe0667 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:51:09.822707 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-e706d687-a35d-424e-962d-318e22fe0667 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:51:09.887549 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:51:09.935596 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e706d687-a35d-424e-962d-318e22fe0667 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:51:09.937069 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e706d687-a35d-424e-962d-318e22fe0667 service neutron] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:51:09.955566 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 668/1319] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 09:51:09 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:51:12.898031 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d743a4f-600f-46f3-9ee3-cfbdfb206eac None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:51:12.898462 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d743a4f-600f-46f3-9ee3-cfbdfb206eac None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:51:12.898590 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d743a4f-600f-46f3-9ee3-cfbdfb206eac None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:51:12.899970 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 652/1320] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:51:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 09:51:12.905144 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6d10f6-e382-4eac-981a-189951b1b398 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:51:12.905548 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6d10f6-e382-4eac-981a-189951b1b398 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:51:12.905710 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6d10f6-e382-4eac-981a-189951b1b398 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:51:12.970404 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-9b6d10f6-e382-4eac-981a-189951b1b398 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:51:12.971612 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-9b6d10f6-e382-4eac-981a-189951b1b398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:51:13.065599 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 669/1321] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:51:12 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:51:13.104830 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d743a4f-600f-46f3-9ee3-cfbdfb206eac None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:51:13.106386 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-9d743a4f-600f-46f3-9ee3-cfbdfb206eac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:51:13.156670 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:51:13.209152 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d743a4f-600f-46f3-9ee3-cfbdfb206eac None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:51:13.280273 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934f03c5-01c2-48cb-9ebb-8e81575b567a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:51:13.280890 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934f03c5-01c2-48cb-9ebb-8e81575b567a service neutron] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:51:13.281178 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934f03c5-01c2-48cb-9ebb-8e81575b567a service neutron] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:51:13.281452 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934f03c5-01c2-48cb-9ebb-8e81575b567a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:51:13.283073 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-934f03c5-01c2-48cb-9ebb-8e81575b567a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:51:13.376386 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:51:13.389708 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934f03c5-01c2-48cb-9ebb-8e81575b567a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:51:13.391152 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934f03c5-01c2-48cb-9ebb-8e81575b567a service neutron] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:51:13.409220 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 653/1322] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 09:51:13 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:51:16.071639 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0994e583-ada6-4961-9ed7-10384ff7b06b None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:51:16.072222 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0994e583-ada6-4961-9ed7-10384ff7b06b None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:51:16.072222 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0994e583-ada6-4961-9ed7-10384ff7b06b None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:51:16.073613 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 670/1323] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:51:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:51:16.078785 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5341d3-781d-49e9-8413-5bac041822ec None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:51:16.079244 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5341d3-781d-49e9-8413-5bac041822ec None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:51:16.079350 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5341d3-781d-49e9-8413-5bac041822ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:51:16.147450 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-3e5341d3-781d-49e9-8413-5bac041822ec None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:51:16.148621 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-3e5341d3-781d-49e9-8413-5bac041822ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:51:16.250590 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 654/1324] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:51:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:51:16.288198 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0994e583-ada6-4961-9ed7-10384ff7b06b None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:51:16.289781 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-0994e583-ada6-4961-9ed7-10384ff7b06b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:51:16.310343 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:51:16.400607 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0994e583-ada6-4961-9ed7-10384ff7b06b None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:51:16.474090 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a3f72f-2b11-4dca-8525-06073e02bf02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:51:16.474706 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a3f72f-2b11-4dca-8525-06073e02bf02 service neutron] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:51:16.474989 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a3f72f-2b11-4dca-8525-06073e02bf02 service neutron] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:51:16.475261 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a3f72f-2b11-4dca-8525-06073e02bf02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:51:16.476922 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-27a3f72f-2b11-4dca-8525-06073e02bf02 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:51:16.547439 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:51:16.587591 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a3f72f-2b11-4dca-8525-06073e02bf02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:51:16.589083 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a3f72f-2b11-4dca-8525-06073e02bf02 service neutron] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:51:16.609349 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 671/1325] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 09:51:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:51:18.643605 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f2a5fc-236f-4985-8c55-e97e40df291c None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:51:18.644065 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f2a5fc-236f-4985-8c55-e97e40df291c None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:51:18.644159 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f2a5fc-236f-4985-8c55-e97e40df291c None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:51:18.646149 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 655/1326] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:51:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:51:18.651128 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deaa055-1cf5-4fe9-ad01-655ce70996a0 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:51:18.651572 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deaa055-1cf5-4fe9-ad01-655ce70996a0 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:51:18.651681 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deaa055-1cf5-4fe9-ad01-655ce70996a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:51:18.717743 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-7deaa055-1cf5-4fe9-ad01-655ce70996a0 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:51:18.718865 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-7deaa055-1cf5-4fe9-ad01-655ce70996a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:51:18.819010 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 672/1327] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:51:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:51:18.856012 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f2a5fc-236f-4985-8c55-e97e40df291c None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:51:18.857609 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-79f2a5fc-236f-4985-8c55-e97e40df291c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:51:18.919719 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:51:18.963450 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f2a5fc-236f-4985-8c55-e97e40df291c None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:51:19.036388 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617f6f1e-afb0-4ca6-ba20-434002fe4b42 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:51:19.037007 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617f6f1e-afb0-4ca6-ba20-434002fe4b42 service neutron] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:51:19.037422 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617f6f1e-afb0-4ca6-ba20-434002fe4b42 service neutron] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:51:19.037696 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617f6f1e-afb0-4ca6-ba20-434002fe4b42 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:51:19.039340 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-617f6f1e-afb0-4ca6-ba20-434002fe4b42 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:51:19.151453 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:51:19.154538 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617f6f1e-afb0-4ca6-ba20-434002fe4b42 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:51:19.156408 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617f6f1e-afb0-4ca6-ba20-434002fe4b42 service neutron] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:51:19.178181 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 656/1328] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 09:51:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:51:21.247761 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb9b09f-3253-459b-a0f9-d1515a597188 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:51:21.248412 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb9b09f-3253-459b-a0f9-d1515a597188 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:51:21.248412 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb9b09f-3253-459b-a0f9-d1515a597188 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:51:21.249628 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 673/1329] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:51:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:51:21.255164 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ce3cff-88db-4af0-abee-9ce4ddf65a8a None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:51:21.255623 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ce3cff-88db-4af0-abee-9ce4ddf65a8a None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:51:21.255797 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ce3cff-88db-4af0-abee-9ce4ddf65a8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:51:21.322252 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-c7ce3cff-88db-4af0-abee-9ce4ddf65a8a None None] MFA Rules not processed for user `6b35b3486c4b4db6a107718cf2807616`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:51:21.323461 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-c7ce3cff-88db-4af0-abee-9ce4ddf65a8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:51:21.422366 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 657/1330] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:51:21 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:51:21.461677 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb9b09f-3253-459b-a0f9-d1515a597188 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:51:21.463327 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-6bb9b09f-3253-459b-a0f9-d1515a597188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:51:21.496768 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:51:21.570401 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb9b09f-3253-459b-a0f9-d1515a597188 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:51:21.642059 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deae9927-0f71-4e28-bb93-a9e4342ab2a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:51:21.642678 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deae9927-0f71-4e28-bb93-a9e4342ab2a8 service neutron] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:51:21.642962 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deae9927-0f71-4e28-bb93-a9e4342ab2a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:51:21.643232 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deae9927-0f71-4e28-bb93-a9e4342ab2a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:51:21.644902 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-deae9927-0f71-4e28-bb93-a9e4342ab2a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:51:21.724691 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:51:21.756171 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deae9927-0f71-4e28-bb93-a9e4342ab2a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:51:21.757671 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deae9927-0f71-4e28-bb93-a9e4342ab2a8 service neutron] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:51:21.775121 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 674/1331] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 09:51:21 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:51:24.021941 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19757ff-28de-418e-86ee-b5468c87ca1f None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:51:24.022469 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19757ff-28de-418e-86ee-b5468c87ca1f None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:51:24.022469 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19757ff-28de-418e-86ee-b5468c87ca1f None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:51:24.023863 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 658/1332] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:51:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:51:24.029258 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274bc706-ff18-49a0-ac0c-f344a272476e None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:51:24.029690 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274bc706-ff18-49a0-ac0c-f344a272476e None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:51:24.029820 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274bc706-ff18-49a0-ac0c-f344a272476e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:51:24.095705 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-274bc706-ff18-49a0-ac0c-f344a272476e None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:51:24.096938 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-274bc706-ff18-49a0-ac0c-f344a272476e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:51:24.189480 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 675/1333] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:51:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:51:24.230454 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19757ff-28de-418e-86ee-b5468c87ca1f None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:51:24.232007 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-a19757ff-28de-418e-86ee-b5468c87ca1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:51:24.303525 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:51:24.338624 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19757ff-28de-418e-86ee-b5468c87ca1f None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:51:24.410758 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b88748-9f32-4515-a9d8-9f932634449c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:51:24.411371 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b88748-9f32-4515-a9d8-9f932634449c service neutron] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:51:24.411651 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b88748-9f32-4515-a9d8-9f932634449c service neutron] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:51:24.411924 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b88748-9f32-4515-a9d8-9f932634449c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:51:24.413566 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-b2b88748-9f32-4515-a9d8-9f932634449c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:51:24.520716 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b88748-9f32-4515-a9d8-9f932634449c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:51:24.522079 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b88748-9f32-4515-a9d8-9f932634449c service neutron] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:51:24.531791 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:51:24.540194 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 659/1334] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 09:51:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:51:26.570226 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9264962-1b6e-4c3f-8073-44fc847b9722 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:51:26.570801 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9264962-1b6e-4c3f-8073-44fc847b9722 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:51:26.570801 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9264962-1b6e-4c3f-8073-44fc847b9722 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:51:26.572138 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 676/1335] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:51:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:51:26.577184 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629150ec-a454-4264-a8b4-31ce9a6d82b9 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:51:26.577629 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629150ec-a454-4264-a8b4-31ce9a6d82b9 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:51:26.577737 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629150ec-a454-4264-a8b4-31ce9a6d82b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:51:26.642677 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-629150ec-a454-4264-a8b4-31ce9a6d82b9 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:51:26.643873 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-629150ec-a454-4264-a8b4-31ce9a6d82b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:51:26.739948 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 660/1336] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:51:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:51:26.779097 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9264962-1b6e-4c3f-8073-44fc847b9722 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:51:26.780667 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-f9264962-1b6e-4c3f-8073-44fc847b9722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:51:26.819764 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:51:26.883555 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9264962-1b6e-4c3f-8073-44fc847b9722 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:51:26.955257 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4204f08-72f4-429c-8d6c-eda84915e441 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:51:26.955868 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4204f08-72f4-429c-8d6c-eda84915e441 service neutron] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:51:26.956153 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4204f08-72f4-429c-8d6c-eda84915e441 service neutron] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:51:26.956457 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4204f08-72f4-429c-8d6c-eda84915e441 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:51:26.958598 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-c4204f08-72f4-429c-8d6c-eda84915e441 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:51:27.040038 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:51:27.065350 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4204f08-72f4-429c-8d6c-eda84915e441 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:51:27.067311 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4204f08-72f4-429c-8d6c-eda84915e441 service neutron] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:51:27.085395 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 677/1337] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 09:51:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:51:29.423695 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5945d3-8cf9-4606-be30-9055164a0c03 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:51:29.424239 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5945d3-8cf9-4606-be30-9055164a0c03 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:51:29.424239 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5945d3-8cf9-4606-be30-9055164a0c03 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:51:29.425752 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 661/1338] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:51:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:51:29.431062 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0319b02c-083c-4fc5-8f5f-a54e2ffc3493 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:51:29.431468 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0319b02c-083c-4fc5-8f5f-a54e2ffc3493 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:51:29.431619 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0319b02c-083c-4fc5-8f5f-a54e2ffc3493 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:51:29.496415 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-0319b02c-083c-4fc5-8f5f-a54e2ffc3493 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:51:29.497613 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-0319b02c-083c-4fc5-8f5f-a54e2ffc3493 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:51:29.593017 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 678/1339] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:51:29 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:51:29.629699 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5945d3-8cf9-4606-be30-9055164a0c03 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:51:29.631259 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-6e5945d3-8cf9-4606-be30-9055164a0c03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:51:29.705524 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:51:29.732919 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5945d3-8cf9-4606-be30-9055164a0c03 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:51:29.805699 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4369b2d2-ff78-451d-97cd-ac169fdb115d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:51:29.806307 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4369b2d2-ff78-451d-97cd-ac169fdb115d service neutron] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:51:29.806587 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4369b2d2-ff78-451d-97cd-ac169fdb115d service neutron] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:51:29.806859 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4369b2d2-ff78-451d-97cd-ac169fdb115d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:51:29.808487 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-4369b2d2-ff78-451d-97cd-ac169fdb115d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:51:29.916797 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4369b2d2-ff78-451d-97cd-ac169fdb115d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:51:29.918226 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4369b2d2-ff78-451d-97cd-ac169fdb115d service neutron] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:51:29.936087 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 662/1340] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 09:51:29 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:51:33.795426 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2386dd8-679d-4415-a823-285fe176cad5 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:51:33.795946 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2386dd8-679d-4415-a823-285fe176cad5 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:51:33.795946 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2386dd8-679d-4415-a823-285fe176cad5 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:51:33.797269 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 679/1341] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:51:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:51:33.803053 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c098d27a-99c4-4964-893e-bbda0667249c None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:51:33.803474 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c098d27a-99c4-4964-893e-bbda0667249c None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:51:33.803614 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c098d27a-99c4-4964-893e-bbda0667249c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:51:33.809123 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:51:33.871422 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-c098d27a-99c4-4964-893e-bbda0667249c None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:51:33.872658 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-c098d27a-99c4-4964-893e-bbda0667249c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:51:33.970269 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 663/1342] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:51:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:51:34.009263 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2386dd8-679d-4415-a823-285fe176cad5 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:51:34.010858 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-f2386dd8-679d-4415-a823-285fe176cad5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:51:34.059466 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:51:34.115703 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2386dd8-679d-4415-a823-285fe176cad5 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:51:34.188952 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be73a6d-4895-4a8b-8089-9323a08600cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:51:34.189564 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be73a6d-4895-4a8b-8089-9323a08600cd service neutron] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:51:34.189844 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be73a6d-4895-4a8b-8089-9323a08600cd service neutron] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:51:34.190117 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be73a6d-4895-4a8b-8089-9323a08600cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:51:34.191746 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-7be73a6d-4895-4a8b-8089-9323a08600cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:51:34.286139 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:51:34.302671 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be73a6d-4895-4a8b-8089-9323a08600cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:51:34.304119 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be73a6d-4895-4a8b-8089-9323a08600cd service neutron] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:51:34.321864 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 680/1343] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 09:51:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:51:36.973800 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6effa594-edf9-4c2a-96ad-2eedde3b2629 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:51:36.974379 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6effa594-edf9-4c2a-96ad-2eedde3b2629 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:51:36.974379 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6effa594-edf9-4c2a-96ad-2eedde3b2629 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:51:36.975742 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 664/1344] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:51:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:51:36.982066 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab143e90-4a18-4d6f-8c7b-d41e570d0d78 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:51:36.982589 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab143e90-4a18-4d6f-8c7b-d41e570d0d78 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:51:36.982710 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab143e90-4a18-4d6f-8c7b-d41e570d0d78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:51:37.059079 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-ab143e90-4a18-4d6f-8c7b-d41e570d0d78 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:51:37.060394 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-ab143e90-4a18-4d6f-8c7b-d41e570d0d78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:51:37.160184 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 681/1345] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:51:36 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:51:37.197581 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6effa594-edf9-4c2a-96ad-2eedde3b2629 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:51:37.199154 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-6effa594-edf9-4c2a-96ad-2eedde3b2629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:51:37.286494 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:51:37.303601 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6effa594-edf9-4c2a-96ad-2eedde3b2629 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:51:37.377390 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8f25bf-e60c-4081-9004-3c4ed856d502 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:51:37.378035 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8f25bf-e60c-4081-9004-3c4ed856d502 service neutron] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:51:37.378593 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8f25bf-e60c-4081-9004-3c4ed856d502 service neutron] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:51:37.378945 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8f25bf-e60c-4081-9004-3c4ed856d502 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:51:37.380915 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-8e8f25bf-e60c-4081-9004-3c4ed856d502 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:51:37.496646 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8f25bf-e60c-4081-9004-3c4ed856d502 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:51:37.498022 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8f25bf-e60c-4081-9004-3c4ed856d502 service neutron] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:51:37.516824 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 665/1346] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 09:51:37 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:51:41.470884 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8014977-ef33-439e-9873-202e891afdac None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:51:41.471423 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8014977-ef33-439e-9873-202e891afdac None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:51:41.471423 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8014977-ef33-439e-9873-202e891afdac None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:51:41.473738 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 682/1347] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:51:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:51:41.478504 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53931f3-ff03-4e14-9323-07ef8a326ecf None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:51:41.478951 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53931f3-ff03-4e14-9323-07ef8a326ecf None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:51:41.479049 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53931f3-ff03-4e14-9323-07ef8a326ecf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:51:41.495107 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:51:41.547127 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-c53931f3-ff03-4e14-9323-07ef8a326ecf None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:51:41.548362 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-c53931f3-ff03-4e14-9323-07ef8a326ecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:51:41.646270 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 666/1348] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:51:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:51:41.684687 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8014977-ef33-439e-9873-202e891afdac None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:51:41.686251 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-e8014977-ef33-439e-9873-202e891afdac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:51:41.761830 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:51:41.815357 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8014977-ef33-439e-9873-202e891afdac None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:51:41.895397 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a37cff-0a51-44c1-a194-1306d9d792ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:51:41.896033 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a37cff-0a51-44c1-a194-1306d9d792ac service neutron] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:51:41.896420 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a37cff-0a51-44c1-a194-1306d9d792ac service neutron] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:51:41.896702 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a37cff-0a51-44c1-a194-1306d9d792ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:51:41.898372 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-d4a37cff-0a51-44c1-a194-1306d9d792ac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:51:42.023788 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:51:42.028055 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a37cff-0a51-44c1-a194-1306d9d792ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:51:42.030023 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a37cff-0a51-44c1-a194-1306d9d792ac service neutron] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:51:42.049944 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 683/1349] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 09:51:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:51:47.099337 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7423fee1-5187-4981-b314-5232899c6ff0 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:51:47.099905 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7423fee1-5187-4981-b314-5232899c6ff0 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:51:47.099905 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7423fee1-5187-4981-b314-5232899c6ff0 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:51:47.119437 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 667/1350] 10.222.0.64 () {54 vars in 921 bytes} [Tue Aug 2 09:51:47 2022] GET /identity => generated 270 bytes in 21 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:51:47.129988 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d79adb-7383-4f73-84ba-0620915bc139 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:51:47.130952 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d79adb-7383-4f73-84ba-0620915bc139 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:51:47.131450 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d79adb-7383-4f73-84ba-0620915bc139 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:51:47.243121 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-01d79adb-7383-4f73-84ba-0620915bc139 None None] MFA Rules not processed for user `b770f073c86b4d88837a8b1f796a8b1d`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:51:47.244463 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-01d79adb-7383-4f73-84ba-0620915bc139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:51:47.365088 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 684/1351] 10.222.0.64 () {58 vars in 1018 bytes} [Tue Aug 2 09:51:47 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:51:51.545651 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2940a73-2f55-4c03-bfa1-6daed2ed51f2 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:51:51.546208 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2940a73-2f55-4c03-bfa1-6daed2ed51f2 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:51:51.546441 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2940a73-2f55-4c03-bfa1-6daed2ed51f2 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:51:51.548029 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 668/1352] 10.222.0.64 () {54 vars in 921 bytes} [Tue Aug 2 09:51:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:51:51.553841 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0804581e-03aa-4222-878b-c290885d3596 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:51:51.554309 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0804581e-03aa-4222-878b-c290885d3596 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:51:51.554633 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0804581e-03aa-4222-878b-c290885d3596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:51:51.616756 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:51:51.642804 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=59 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:51:51.645227 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-0804581e-03aa-4222-878b-c290885d3596 None None] MFA Rules not processed for user `b770f073c86b4d88837a8b1f796a8b1d`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:51:51.646843 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-0804581e-03aa-4222-878b-c290885d3596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:51:51.762893 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 685/1353] 10.222.0.64 () {58 vars in 1018 bytes} [Tue Aug 2 09:51:51 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:51:53.064530 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73513ce4-9743-43f1-8ffd-2bd0732d2d52 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:51:53.065056 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73513ce4-9743-43f1-8ffd-2bd0732d2d52 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:51:53.065370 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73513ce4-9743-43f1-8ffd-2bd0732d2d52 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:51:53.066814 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 669/1354] 10.222.0.64 () {54 vars in 921 bytes} [Tue Aug 2 09:51:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:51:53.071831 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b32d40d-91e8-48f2-a702-d0f51467449c None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:51:53.072288 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b32d40d-91e8-48f2-a702-d0f51467449c None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:51:53.072426 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b32d40d-91e8-48f2-a702-d0f51467449c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:51:53.150896 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-2b32d40d-91e8-48f2-a702-d0f51467449c None None] MFA Rules not processed for user `b770f073c86b4d88837a8b1f796a8b1d`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:51:53.152019 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-2b32d40d-91e8-48f2-a702-d0f51467449c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:51:53.195445 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:51:53.259382 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 686/1355] 10.222.0.64 () {58 vars in 1018 bytes} [Tue Aug 2 09:51:53 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:51:56.594376 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa36ccd1-4352-4661-91a0-6cba45b7b731 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:51:56.595240 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa36ccd1-4352-4661-91a0-6cba45b7b731 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:51:56.595665 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa36ccd1-4352-4661-91a0-6cba45b7b731 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:51:56.598143 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 670/1356] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:51:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:51:56.603607 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2203d2e-5319-4c31-b37f-38215a565a6d None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:51:56.604433 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2203d2e-5319-4c31-b37f-38215a565a6d None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:51:56.604888 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2203d2e-5319-4c31-b37f-38215a565a6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:51:56.694224 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-e2203d2e-5319-4c31-b37f-38215a565a6d None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:51:56.695430 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-e2203d2e-5319-4c31-b37f-38215a565a6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:51:56.802726 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 687/1357] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:51:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:51:56.857760 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23383f83-c0d7-4529-90d0-0382702daf2c None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:51:56.858435 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23383f83-c0d7-4529-90d0-0382702daf2c None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:51:56.858966 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23383f83-c0d7-4529-90d0-0382702daf2c None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:51:56.860526 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 671/1358] 10.222.0.64 () {54 vars in 962 bytes} [Tue Aug 2 09:51:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:51:56.868170 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efc0280-54ee-473b-97c0-b3bed85cb3d7 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:51:56.869506 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efc0280-54ee-473b-97c0-b3bed85cb3d7 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:51:56.870219 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efc0280-54ee-473b-97c0-b3bed85cb3d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:51:56.892291 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:51:56.962930 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-3efc0280-54ee-473b-97c0-b3bed85cb3d7 None None] MFA Rules not processed for user `99655e1f9ca24472a86cd26b0dbd1fc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:51:56.964138 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-3efc0280-54ee-473b-97c0-b3bed85cb3d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:51:57.093163 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 688/1359] 10.222.0.64 () {58 vars in 1059 bytes} [Tue Aug 2 09:51:56 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:51:57.103047 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23383f83-c0d7-4529-90d0-0382702daf2c None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:51:57.104889 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-23383f83-c0d7-4529-90d0-0382702daf2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:51:57.255346 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23383f83-c0d7-4529-90d0-0382702daf2c None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:51:57.347705 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:51:57.390538 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14165357-4bab-4f96-9bbc-8575372fd3ea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99655e1f9ca24472a86cd26b0dbd1fc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:51:57.391536 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14165357-4bab-4f96-9bbc-8575372fd3ea service nova] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:51:57.391782 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14165357-4bab-4f96-9bbc-8575372fd3ea service nova] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:51:57.392188 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14165357-4bab-4f96-9bbc-8575372fd3ea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:51:57.395685 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-14165357-4bab-4f96-9bbc-8575372fd3ea service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:51:57.490608 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14165357-4bab-4f96-9bbc-8575372fd3ea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:51:57.492426 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14165357-4bab-4f96-9bbc-8575372fd3ea service nova] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:51:57.511242 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 672/1360] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 09:51:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:51:59.726537 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebed3fab-d04c-4f18-953c-92255091be60 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:51:59.727024 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebed3fab-d04c-4f18-953c-92255091be60 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:51:59.727024 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebed3fab-d04c-4f18-953c-92255091be60 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:51:59.732161 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 689/1361] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:51:59 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:51:59.733599 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde8cc5b-4824-4ccf-9f8f-9e6d05051ee2 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:51:59.733976 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde8cc5b-4824-4ccf-9f8f-9e6d05051ee2 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:51:59.734444 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde8cc5b-4824-4ccf-9f8f-9e6d05051ee2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:51:59.814528 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-fde8cc5b-4824-4ccf-9f8f-9e6d05051ee2 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:51:59.816426 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-fde8cc5b-4824-4ccf-9f8f-9e6d05051ee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:51:59.857807 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:51:59.925531 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 673/1362] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:51:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:51:59.974285 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebed3fab-d04c-4f18-953c-92255091be60 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:51:59.975925 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-ebed3fab-d04c-4f18-953c-92255091be60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:00.052781 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:52:00.145776 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebed3fab-d04c-4f18-953c-92255091be60 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:00.264044 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1de696-eed8-42cf-ab5b-98cb419d0746 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99655e1f9ca24472a86cd26b0dbd1fc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:00.264699 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1de696-eed8-42cf-ab5b-98cb419d0746 service nova] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:00.264995 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1de696-eed8-42cf-ab5b-98cb419d0746 service nova] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:00.265287 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1de696-eed8-42cf-ab5b-98cb419d0746 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:00.266873 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-cf1de696-eed8-42cf-ab5b-98cb419d0746 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:00.291421 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:52:00.368970 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1de696-eed8-42cf-ab5b-98cb419d0746 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:00.370579 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1de696-eed8-42cf-ab5b-98cb419d0746 service nova] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:00.391719 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 690/1363] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 09:51:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:52:02.712392 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fd0edb-5245-4e50-b3b0-2bfbe95481d4 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:02.712940 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fd0edb-5245-4e50-b3b0-2bfbe95481d4 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:02.712940 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fd0edb-5245-4e50-b3b0-2bfbe95481d4 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:02.714313 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 674/1364] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:52:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:52:02.720657 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77412dce-6f0f-4a89-b547-ddcaff6d73bb None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:02.721181 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77412dce-6f0f-4a89-b547-ddcaff6d73bb None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:02.721295 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77412dce-6f0f-4a89-b547-ddcaff6d73bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:02.795658 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-77412dce-6f0f-4a89-b547-ddcaff6d73bb None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:52:02.797195 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-77412dce-6f0f-4a89-b547-ddcaff6d73bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:02.918631 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:52:02.922093 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 691/1365] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:52:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:52:02.971764 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e94466-0103-4dad-90ec-fa6f3a298dc4 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:02.972046 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e94466-0103-4dad-90ec-fa6f3a298dc4 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:02.972533 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e94466-0103-4dad-90ec-fa6f3a298dc4 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:02.974348 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 675/1366] 10.222.0.64 () {54 vars in 962 bytes} [Tue Aug 2 09:52:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:52:02.982004 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55084d60-6520-47b1-9962-1579ce289ae6 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:02.982300 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55084d60-6520-47b1-9962-1579ce289ae6 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:02.982583 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55084d60-6520-47b1-9962-1579ce289ae6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:03.076599 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-55084d60-6520-47b1-9962-1579ce289ae6 None None] MFA Rules not processed for user `99655e1f9ca24472a86cd26b0dbd1fc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:52:03.077846 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-55084d60-6520-47b1-9962-1579ce289ae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:03.203077 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 692/1367] 10.222.0.64 () {58 vars in 1059 bytes} [Tue Aug 2 09:52:02 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:52:03.211227 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e94466-0103-4dad-90ec-fa6f3a298dc4 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:03.212782 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-42e94466-0103-4dad-90ec-fa6f3a298dc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:03.369498 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e94466-0103-4dad-90ec-fa6f3a298dc4 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:03.392350 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:52:03.495475 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608f6b28-a0a8-4311-b855-1a4f316f8007 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99655e1f9ca24472a86cd26b0dbd1fc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:03.496082 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608f6b28-a0a8-4311-b855-1a4f316f8007 service nova] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:03.496418 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608f6b28-a0a8-4311-b855-1a4f316f8007 service nova] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:03.496703 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608f6b28-a0a8-4311-b855-1a4f316f8007 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:03.498345 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-608f6b28-a0a8-4311-b855-1a4f316f8007 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:03.598848 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608f6b28-a0a8-4311-b855-1a4f316f8007 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:03.600509 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608f6b28-a0a8-4311-b855-1a4f316f8007 service nova] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:03.620164 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 676/1368] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 09:52:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 09:52:04.709080 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ccece8-ce64-4d2b-8640-d9a48ad45668 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:04.709602 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ccece8-ce64-4d2b-8640-d9a48ad45668 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:04.709730 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ccece8-ce64-4d2b-8640-d9a48ad45668 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:04.711276 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 693/1369] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:52:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:52:04.717048 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a968b148-e149-45bd-bf03-63b5c6657008 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:04.717454 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a968b148-e149-45bd-bf03-63b5c6657008 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:04.717620 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a968b148-e149-45bd-bf03-63b5c6657008 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:04.754188 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:52:04.796026 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-a968b148-e149-45bd-bf03-63b5c6657008 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:52:04.797285 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-a968b148-e149-45bd-bf03-63b5c6657008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:04.921157 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 677/1370] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:52:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:52:05.260708 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07937424-75e0-47fd-a399-3852ee0e23d7 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:05.260997 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07937424-75e0-47fd-a399-3852ee0e23d7 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:05.261274 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07937424-75e0-47fd-a399-3852ee0e23d7 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:05.262788 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 694/1371] 10.222.0.64 () {54 vars in 964 bytes} [Tue Aug 2 09:52:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:52:05.268706 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456984e6-9f58-4d7f-97b1-982b39ac3f3e None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:05.269243 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456984e6-9f58-4d7f-97b1-982b39ac3f3e None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:05.269853 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456984e6-9f58-4d7f-97b1-982b39ac3f3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:05.351932 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-456984e6-9f58-4d7f-97b1-982b39ac3f3e None None] MFA Rules not processed for user `10d7de62b9ee4039bdc79627ac7fb499`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:52:05.353039 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-456984e6-9f58-4d7f-97b1-982b39ac3f3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:05.403061 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:52:05.435483 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 678/1372] 10.222.0.64 () {58 vars in 1061 bytes} [Tue Aug 2 09:52:05 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:52:05.445041 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07937424-75e0-47fd-a399-3852ee0e23d7 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:05.446706 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-07937424-75e0-47fd-a399-3852ee0e23d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:05.499761 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:52:05.560175 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07937424-75e0-47fd-a399-3852ee0e23d7 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:05.639124 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e044d7-aa1e-4eb5-a671-0417c064973c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d7de62b9ee4039bdc79627ac7fb499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:05.639897 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e044d7-aa1e-4eb5-a671-0417c064973c service cinder] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:05.640303 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e044d7-aa1e-4eb5-a671-0417c064973c service cinder] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:05.640703 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e044d7-aa1e-4eb5-a671-0417c064973c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:05.642415 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-b5e044d7-aa1e-4eb5-a671-0417c064973c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:05.682674 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50ed0f9-7ee1-4f5f-a8c6-08d74da657a7 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:05.683011 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50ed0f9-7ee1-4f5f-a8c6-08d74da657a7 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:05.683249 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50ed0f9-7ee1-4f5f-a8c6-08d74da657a7 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:05.684795 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 679/1373] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:52:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:52:05.689356 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77e35fb-adef-4d9d-993c-598e923acb6a None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:05.690209 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77e35fb-adef-4d9d-993c-598e923acb6a None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:05.690815 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77e35fb-adef-4d9d-993c-598e923acb6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:05.749942 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:52:05.760145 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-d77e35fb-adef-4d9d-993c-598e923acb6a None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:52:05.761457 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-d77e35fb-adef-4d9d-993c-598e923acb6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:05.764214 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e044d7-aa1e-4eb5-a671-0417c064973c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:05.765688 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e044d7-aa1e-4eb5-a671-0417c064973c service cinder] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:05.784927 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 695/1374] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 09:52:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:52:05.879409 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 680/1375] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:52:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:52:05.928654 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e044d7-aa1e-4eb5-a671-0417c064973c service cinder] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:05.929940 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-b5e044d7-aa1e-4eb5-a671-0417c064973c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:06.035500 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e044d7-aa1e-4eb5-a671-0417c064973c service cinder] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:06.153789 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:52:06.162272 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50537759-5934-400e-b7ff-e988f164d388 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99655e1f9ca24472a86cd26b0dbd1fc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:06.162905 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50537759-5934-400e-b7ff-e988f164d388 service nova] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:06.163166 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50537759-5934-400e-b7ff-e988f164d388 service nova] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:06.163415 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50537759-5934-400e-b7ff-e988f164d388 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:06.165067 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-50537759-5934-400e-b7ff-e988f164d388 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:06.273482 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50537759-5934-400e-b7ff-e988f164d388 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:06.275074 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50537759-5934-400e-b7ff-e988f164d388 service nova] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:06.293204 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 696/1376] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 09:52:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:52:08.327384 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57d4b4e-d954-4175-971d-6db5da8a41fa None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:08.327929 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57d4b4e-d954-4175-971d-6db5da8a41fa None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:08.328036 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57d4b4e-d954-4175-971d-6db5da8a41fa None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:08.329377 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 681/1377] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:52:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:52:08.335208 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a2578f-4d73-45c7-b29b-98f781f13589 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:08.335635 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a2578f-4d73-45c7-b29b-98f781f13589 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:08.335769 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a2578f-4d73-45c7-b29b-98f781f13589 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:08.422791 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-19a2578f-4d73-45c7-b29b-98f781f13589 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:52:08.424009 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-19a2578f-4d73-45c7-b29b-98f781f13589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:08.485419 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:52:08.525986 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 697/1378] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:52:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:52:08.570023 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57d4b4e-d954-4175-971d-6db5da8a41fa None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:08.571592 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-f57d4b4e-d954-4175-971d-6db5da8a41fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:08.633160 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:52:08.726555 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57d4b4e-d954-4175-971d-6db5da8a41fa None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:08.872895 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bb4894-61b7-4887-9dfa-8ba9f6bf34c9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99655e1f9ca24472a86cd26b0dbd1fc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:08.873424 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bb4894-61b7-4887-9dfa-8ba9f6bf34c9 service nova] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:08.873708 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bb4894-61b7-4887-9dfa-8ba9f6bf34c9 service nova] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:08.873981 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bb4894-61b7-4887-9dfa-8ba9f6bf34c9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:08.875611 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-07bb4894-61b7-4887-9dfa-8ba9f6bf34c9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:08.893428 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:52:08.966771 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bb4894-61b7-4887-9dfa-8ba9f6bf34c9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:08.968405 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bb4894-61b7-4887-9dfa-8ba9f6bf34c9 service nova] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:08.986615 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 682/1379] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 09:52:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:52:11.040002 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40b2b84-c72b-4bf6-aee8-ab049d4690b5 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:11.040616 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40b2b84-c72b-4bf6-aee8-ab049d4690b5 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:11.040616 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40b2b84-c72b-4bf6-aee8-ab049d4690b5 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:11.042219 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 698/1380] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:52:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:52:11.047370 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaed0f2-6c8b-4689-b1d6-4c3bcf35c031 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:11.047803 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaed0f2-6c8b-4689-b1d6-4c3bcf35c031 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:11.047916 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaed0f2-6c8b-4689-b1d6-4c3bcf35c031 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:11.117547 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-ceaed0f2-6c8b-4689-b1d6-4c3bcf35c031 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:52:11.118799 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-ceaed0f2-6c8b-4689-b1d6-4c3bcf35c031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:11.211783 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:52:11.227098 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 683/1381] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:52:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:52:11.271821 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40b2b84-c72b-4bf6-aee8-ab049d4690b5 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:11.273358 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-a40b2b84-c72b-4bf6-aee8-ab049d4690b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:11.445647 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40b2b84-c72b-4bf6-aee8-ab049d4690b5 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:11.497054 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:52:11.564876 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75e58dd-ca4b-4fc4-bfc4-1c094c02f9f9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99655e1f9ca24472a86cd26b0dbd1fc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:11.565561 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75e58dd-ca4b-4fc4-bfc4-1c094c02f9f9 service nova] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:11.565905 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75e58dd-ca4b-4fc4-bfc4-1c094c02f9f9 service nova] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:11.566238 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75e58dd-ca4b-4fc4-bfc4-1c094c02f9f9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:11.568047 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-e75e58dd-ca4b-4fc4-bfc4-1c094c02f9f9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:11.669717 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75e58dd-ca4b-4fc4-bfc4-1c094c02f9f9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:11.671340 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75e58dd-ca4b-4fc4-bfc4-1c094c02f9f9 service nova] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:11.691019 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 699/1382] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 09:52:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:52:13.723661 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c090f7f4-588f-48cd-96f9-0efc344866a2 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:13.724228 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c090f7f4-588f-48cd-96f9-0efc344866a2 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:13.724429 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c090f7f4-588f-48cd-96f9-0efc344866a2 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:13.726319 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 684/1383] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:52:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:52:13.731339 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c9e768-1178-420c-8b50-2b43b7b76782 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:13.731795 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c9e768-1178-420c-8b50-2b43b7b76782 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:13.731936 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c9e768-1178-420c-8b50-2b43b7b76782 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:13.803423 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-f3c9e768-1178-420c-8b50-2b43b7b76782 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:52:13.804677 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-f3c9e768-1178-420c-8b50-2b43b7b76782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:13.806774 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:52:13.912166 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 700/1384] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:52:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:52:13.955905 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c090f7f4-588f-48cd-96f9-0efc344866a2 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:13.957930 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-c090f7f4-588f-48cd-96f9-0efc344866a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:14.121200 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c090f7f4-588f-48cd-96f9-0efc344866a2 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:14.194268 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:52:14.252915 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157b8c5a-09bc-4f1c-87c1-6e51c12cb10f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99655e1f9ca24472a86cd26b0dbd1fc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:14.253617 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157b8c5a-09bc-4f1c-87c1-6e51c12cb10f service nova] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:14.253944 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157b8c5a-09bc-4f1c-87c1-6e51c12cb10f service nova] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:14.254266 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157b8c5a-09bc-4f1c-87c1-6e51c12cb10f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:14.256175 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-157b8c5a-09bc-4f1c-87c1-6e51c12cb10f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:14.360003 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157b8c5a-09bc-4f1c-87c1-6e51c12cb10f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:14.362691 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157b8c5a-09bc-4f1c-87c1-6e51c12cb10f service nova] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:14.385767 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 685/1385] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 09:52:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:52:16.492931 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231e4978-484d-4693-bf0b-bb5f2d3bc88f None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:16.493485 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231e4978-484d-4693-bf0b-bb5f2d3bc88f None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:16.493830 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231e4978-484d-4693-bf0b-bb5f2d3bc88f None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:16.496385 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 701/1386] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:52:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:52:16.501844 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fb2a2f-de3c-432d-8bee-6ad6c7f5c2e4 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:16.502358 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fb2a2f-de3c-432d-8bee-6ad6c7f5c2e4 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:16.502358 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fb2a2f-de3c-432d-8bee-6ad6c7f5c2e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:16.569397 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-43fb2a2f-de3c-432d-8bee-6ad6c7f5c2e4 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:52:16.570599 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-43fb2a2f-de3c-432d-8bee-6ad6c7f5c2e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:16.597906 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:52:16.697901 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 686/1387] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:52:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:52:16.744092 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231e4978-484d-4693-bf0b-bb5f2d3bc88f None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:16.746913 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-231e4978-484d-4693-bf0b-bb5f2d3bc88f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:16.915486 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:52:16.940816 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231e4978-484d-4693-bf0b-bb5f2d3bc88f None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:17.074030 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e27a0e7-9d4e-4f8c-ab6a-f758543ff5c5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99655e1f9ca24472a86cd26b0dbd1fc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:17.074738 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e27a0e7-9d4e-4f8c-ab6a-f758543ff5c5 service nova] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:17.075070 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e27a0e7-9d4e-4f8c-ab6a-f758543ff5c5 service nova] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:17.075393 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e27a0e7-9d4e-4f8c-ab6a-f758543ff5c5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:17.077324 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-4e27a0e7-9d4e-4f8c-ab6a-f758543ff5c5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:17.172769 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e27a0e7-9d4e-4f8c-ab6a-f758543ff5c5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:17.174561 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e27a0e7-9d4e-4f8c-ab6a-f758543ff5c5 service nova] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:17.179927 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:52:17.194606 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 702/1388] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 09:52:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:52:19.540876 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f18f67-45d5-44a0-be31-76b8f2249384 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:19.541427 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f18f67-45d5-44a0-be31-76b8f2249384 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:19.541427 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f18f67-45d5-44a0-be31-76b8f2249384 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:19.542797 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 687/1389] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:52:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:52:19.548052 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606f6529-dd14-4818-8e37-d95602245688 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:19.548575 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606f6529-dd14-4818-8e37-d95602245688 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:19.548687 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606f6529-dd14-4818-8e37-d95602245688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:19.617244 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-606f6529-dd14-4818-8e37-d95602245688 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:52:19.618443 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-606f6529-dd14-4818-8e37-d95602245688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:19.716148 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 703/1390] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:52:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:52:19.759982 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f18f67-45d5-44a0-be31-76b8f2249384 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:19.761609 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-09f18f67-45d5-44a0-be31-76b8f2249384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:19.911021 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f18f67-45d5-44a0-be31-76b8f2249384 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:19.920892 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:52:20.030076 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24958a58-c93d-4d78-a959-d4ff5244198b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99655e1f9ca24472a86cd26b0dbd1fc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:20.030699 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24958a58-c93d-4d78-a959-d4ff5244198b service nova] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:20.030988 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24958a58-c93d-4d78-a959-d4ff5244198b service nova] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:20.031267 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24958a58-c93d-4d78-a959-d4ff5244198b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:20.033064 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-24958a58-c93d-4d78-a959-d4ff5244198b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:20.122927 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24958a58-c93d-4d78-a959-d4ff5244198b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:20.124562 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24958a58-c93d-4d78-a959-d4ff5244198b service nova] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:20.143204 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 688/1391] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 09:52:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:52:21.461850 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170ec6b8-849e-4d07-b960-2d9c1760f62e None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:21.462463 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170ec6b8-849e-4d07-b960-2d9c1760f62e None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:21.462463 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170ec6b8-849e-4d07-b960-2d9c1760f62e None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:21.463767 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 704/1392] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:52:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:52:21.469107 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1c61e9-5340-44dd-bae6-d57accd691de None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:21.469542 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1c61e9-5340-44dd-bae6-d57accd691de None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:21.469697 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1c61e9-5340-44dd-bae6-d57accd691de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:21.485477 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:52:21.535837 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-dd1c61e9-5340-44dd-bae6-d57accd691de None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:52:21.537086 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-dd1c61e9-5340-44dd-bae6-d57accd691de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:21.633003 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 689/1393] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:52:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:52:21.688574 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d6ca5f-b34a-44c9-8a57-4487044f40c7 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:21.689410 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d6ca5f-b34a-44c9-8a57-4487044f40c7 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:21.689691 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d6ca5f-b34a-44c9-8a57-4487044f40c7 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:21.691042 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 705/1394] 10.222.0.64 () {54 vars in 966 bytes} [Tue Aug 2 09:52:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:52:21.695999 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0b4632-518d-482b-bcd4-725927cbdb95 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:21.696316 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0b4632-518d-482b-bcd4-725927cbdb95 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:21.696599 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0b4632-518d-482b-bcd4-725927cbdb95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:21.771995 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-8f0b4632-518d-482b-bcd4-725927cbdb95 None None] MFA Rules not processed for user `04fb1dbe260a4721aaa0bd0493260d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:52:21.773137 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-8f0b4632-518d-482b-bcd4-725927cbdb95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:21.809120 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:52:21.852219 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 690/1395] 10.222.0.64 () {58 vars in 1063 bytes} [Tue Aug 2 09:52:21 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:52:21.859733 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d6ca5f-b34a-44c9-8a57-4487044f40c7 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:21.861845 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-79d6ca5f-b34a-44c9-8a57-4487044f40c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:21.938439 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:52:21.969675 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d6ca5f-b34a-44c9-8a57-4487044f40c7 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:22.044323 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae09e4dd-cc23-49bd-9321-3d238555e23c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04fb1dbe260a4721aaa0bd0493260d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:22.044850 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae09e4dd-cc23-49bd-9321-3d238555e23c service glance] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:22.045130 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae09e4dd-cc23-49bd-9321-3d238555e23c service glance] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:22.045403 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae09e4dd-cc23-49bd-9321-3d238555e23c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:22.047017 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-ae09e4dd-cc23-49bd-9321-3d238555e23c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:22.156208 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae09e4dd-cc23-49bd-9321-3d238555e23c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:22.157674 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae09e4dd-cc23-49bd-9321-3d238555e23c service glance] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:22.172841 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:52:22.177399 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93d7305-573c-4626-8176-e780d3fa6b1a None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:22.177692 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93d7305-573c-4626-8176-e780d3fa6b1a None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:22.178026 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93d7305-573c-4626-8176-e780d3fa6b1a None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:22.178675 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 706/1396] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 09:52:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:52:22.179445 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 691/1397] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:52:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 09:52:22.184640 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073c0525-22fa-4191-ab05-f8a1f0e25630 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:22.184932 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073c0525-22fa-4191-ab05-f8a1f0e25630 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:22.185214 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073c0525-22fa-4191-ab05-f8a1f0e25630 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:22.249583 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-073c0525-22fa-4191-ab05-f8a1f0e25630 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:52:22.250773 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-073c0525-22fa-4191-ab05-f8a1f0e25630 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:22.347644 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 707/1398] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:52:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:52:22.392100 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93d7305-573c-4626-8176-e780d3fa6b1a None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:22.393717 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-c93d7305-573c-4626-8176-e780d3fa6b1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:22.558440 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93d7305-573c-4626-8176-e780d3fa6b1a None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:22.601879 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:52:22.698518 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c011088-dbdd-4453-a4b8-277004e681d6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99655e1f9ca24472a86cd26b0dbd1fc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:22.699145 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c011088-dbdd-4453-a4b8-277004e681d6 service nova] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:22.699670 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c011088-dbdd-4453-a4b8-277004e681d6 service nova] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:22.700008 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c011088-dbdd-4453-a4b8-277004e681d6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:22.701781 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-6c011088-dbdd-4453-a4b8-277004e681d6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:22.799720 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c011088-dbdd-4453-a4b8-277004e681d6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:22.801430 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c011088-dbdd-4453-a4b8-277004e681d6 service nova] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:22.821775 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 692/1399] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 09:52:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:52:22.943377 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311a78de-2fbf-45af-a7ee-65173a9ff771 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:22.943954 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311a78de-2fbf-45af-a7ee-65173a9ff771 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:22.943954 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311a78de-2fbf-45af-a7ee-65173a9ff771 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:23.027608 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-311a78de-2fbf-45af-a7ee-65173a9ff771 None None] MFA Rules not processed for user `5338bc8c594e4bc9bc41019226bf8d87`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:52:23.028983 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-311a78de-2fbf-45af-a7ee-65173a9ff771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:23.063191 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:52:23.123896 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 708/1400] 10.222.0.64 () {58 vars in 1009 bytes} [Tue Aug 2 09:52:22 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:52:23.169930 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886c089a-cd11-4bb5-9980-85fb886ee908 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:23.170226 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886c089a-cd11-4bb5-9980-85fb886ee908 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:23.170533 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886c089a-cd11-4bb5-9980-85fb886ee908 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:23.171911 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 693/1401] 10.222.0.64 () {54 vars in 943 bytes} [Tue Aug 2 09:52:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:52:23.178639 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a7f01e-c844-4c5a-806f-55c24d0b32dd None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:23.179003 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a7f01e-c844-4c5a-806f-55c24d0b32dd None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:23.179348 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a7f01e-c844-4c5a-806f-55c24d0b32dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:23.255910 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-89a7f01e-c844-4c5a-806f-55c24d0b32dd None None] MFA Rules not processed for user `6a371e3df3764a4cbb1b8368aa74bd65`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:52:23.257150 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-89a7f01e-c844-4c5a-806f-55c24d0b32dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:23.366243 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 709/1402] 10.222.0.64 () {58 vars in 1040 bytes} [Tue Aug 2 09:52:23 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:52:23.376049 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886c089a-cd11-4bb5-9980-85fb886ee908 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:23.378113 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-886c089a-cd11-4bb5-9980-85fb886ee908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:23.437369 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:52:23.524498 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886c089a-cd11-4bb5-9980-85fb886ee908 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:23.628635 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2554e6b-4ec4-4338-a815-20018f253bc1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a371e3df3764a4cbb1b8368aa74bd65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:23.629290 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2554e6b-4ec4-4338-a815-20018f253bc1 service swift] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:23.629434 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2554e6b-4ec4-4338-a815-20018f253bc1 service swift] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:23.629705 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2554e6b-4ec4-4338-a815-20018f253bc1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:23.631390 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-e2554e6b-4ec4-4338-a815-20018f253bc1 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:23.660895 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:52:23.714200 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2554e6b-4ec4-4338-a815-20018f253bc1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:23.716078 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2554e6b-4ec4-4338-a815-20018f253bc1 service swift] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:23.723999 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 694/1403] 10.222.0.64 () {60 vars in 1384 bytes} [Tue Aug 2 09:52:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 02 09:52:24.869741 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e29d72-68f8-444d-a70b-96357a30e2fb None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:24.870339 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e29d72-68f8-444d-a70b-96357a30e2fb None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:24.870339 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e29d72-68f8-444d-a70b-96357a30e2fb None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:24.871658 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 710/1404] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:52:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:52:24.877054 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cb97a6-3256-404b-b332-8eac7dfec534 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:24.877513 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cb97a6-3256-404b-b332-8eac7dfec534 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:24.877606 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cb97a6-3256-404b-b332-8eac7dfec534 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:24.952967 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-03cb97a6-3256-404b-b332-8eac7dfec534 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:52:24.954141 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-03cb97a6-3256-404b-b332-8eac7dfec534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:25.075125 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 695/1405] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:52:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:52:25.127120 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e29d72-68f8-444d-a70b-96357a30e2fb None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:25.127120 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-c3e29d72-68f8-444d-a70b-96357a30e2fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:25.129810 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:52:25.318205 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e29d72-68f8-444d-a70b-96357a30e2fb None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:25.402718 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:52:25.443903 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445fda81-5bcc-4a95-9009-b91bc3070fec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99655e1f9ca24472a86cd26b0dbd1fc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:25.444667 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445fda81-5bcc-4a95-9009-b91bc3070fec service nova] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:25.445026 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445fda81-5bcc-4a95-9009-b91bc3070fec service nova] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:25.445375 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445fda81-5bcc-4a95-9009-b91bc3070fec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:25.447039 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-445fda81-5bcc-4a95-9009-b91bc3070fec service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:25.548509 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445fda81-5bcc-4a95-9009-b91bc3070fec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:25.550159 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445fda81-5bcc-4a95-9009-b91bc3070fec service nova] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:25.569460 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 711/1406] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 09:52:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:52:27.600871 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b04472-dd51-4b68-9437-5bf829c9f87d None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:27.601448 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b04472-dd51-4b68-9437-5bf829c9f87d None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:27.601448 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b04472-dd51-4b68-9437-5bf829c9f87d None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:27.606516 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 696/1407] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:52:27 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:52:27.607892 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7055af89-b3a8-483c-aa0d-91997893e84f None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:27.608475 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7055af89-b3a8-483c-aa0d-91997893e84f None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:27.608786 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7055af89-b3a8-483c-aa0d-91997893e84f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:27.694862 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-7055af89-b3a8-483c-aa0d-91997893e84f None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:52:27.696490 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-7055af89-b3a8-483c-aa0d-91997893e84f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:27.759215 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:52:27.836362 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 712/1408] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:52:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:52:27.882820 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b04472-dd51-4b68-9437-5bf829c9f87d None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:27.886985 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-20b04472-dd51-4b68-9437-5bf829c9f87d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:27.912167 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:52:28.059237 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b04472-dd51-4b68-9437-5bf829c9f87d None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:28.166260 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:52:28.187440 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf5f14c-2cc1-4660-acba-e954951618da service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99655e1f9ca24472a86cd26b0dbd1fc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:28.188161 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf5f14c-2cc1-4660-acba-e954951618da service nova] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:28.189087 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf5f14c-2cc1-4660-acba-e954951618da service nova] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:28.191425 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf5f14c-2cc1-4660-acba-e954951618da service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:28.191425 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-acf5f14c-2cc1-4660-acba-e954951618da service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:28.287151 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf5f14c-2cc1-4660-acba-e954951618da service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:28.289086 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf5f14c-2cc1-4660-acba-e954951618da service nova] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:28.311093 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 697/1409] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 09:52:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:52:37.835191 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=26 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:52:38.308811 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=52 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:52:45.654549 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce2dd80-61a2-4fb0-a176-7ae0c7bb3597 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:45.655120 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce2dd80-61a2-4fb0-a176-7ae0c7bb3597 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:45.655120 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce2dd80-61a2-4fb0-a176-7ae0c7bb3597 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:45.657127 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 713/1410] 10.222.0.64 () {54 vars in 918 bytes} [Tue Aug 2 09:52:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:52:45.662339 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8633a3-872d-40c0-896d-3e38cc534e21 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:45.662746 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8633a3-872d-40c0-896d-3e38cc534e21 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:45.662894 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8633a3-872d-40c0-896d-3e38cc534e21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:45.737824 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-cd8633a3-872d-40c0-896d-3e38cc534e21 None None] MFA Rules not processed for user `b770f073c86b4d88837a8b1f796a8b1d`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:52:45.739034 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-cd8633a3-872d-40c0-896d-3e38cc534e21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:45.843073 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 698/1411] 10.222.0.64 () {58 vars in 1015 bytes} [Tue Aug 2 09:52:45 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:52:45.857519 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb46eea5-ce43-478d-93db-b91cc09efc15 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:45.857811 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb46eea5-ce43-478d-93db-b91cc09efc15 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:45.858092 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb46eea5-ce43-478d-93db-b91cc09efc15 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:45.859432 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 714/1412] 10.222.0.64 () {54 vars in 918 bytes} [Tue Aug 2 09:52:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:52:45.863976 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252bc32c-7e7b-4e68-8557-975bd7c569b3 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:45.864292 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252bc32c-7e7b-4e68-8557-975bd7c569b3 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:45.864600 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252bc32c-7e7b-4e68-8557-975bd7c569b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:45.935936 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:52:45.941145 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-252bc32c-7e7b-4e68-8557-975bd7c569b3 None None] MFA Rules not processed for user `99655e1f9ca24472a86cd26b0dbd1fc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:52:45.942934 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-252bc32c-7e7b-4e68-8557-975bd7c569b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:46.056952 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 699/1413] 10.222.0.64 () {58 vars in 1015 bytes} [Tue Aug 2 09:52:45 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:52:48.520929 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e410febe-bbdb-4f0e-b5f7-10a88a0bce5b None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:48.521445 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e410febe-bbdb-4f0e-b5f7-10a88a0bce5b None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:48.521445 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e410febe-bbdb-4f0e-b5f7-10a88a0bce5b None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:48.522747 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 715/1414] 127.0.0.1 () {54 vars in 916 bytes} [Tue Aug 2 09:52:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:52:48.528040 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e58b10-2f55-4e54-bab8-a139169e2f06 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:48.528472 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e58b10-2f55-4e54-bab8-a139169e2f06 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:48.528645 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e58b10-2f55-4e54-bab8-a139169e2f06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:48.594308 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-b3e58b10-2f55-4e54-bab8-a139169e2f06 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:52:48.595504 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-b3e58b10-2f55-4e54-bab8-a139169e2f06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:48.654918 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:52:48.697208 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 700/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Aug 2 09:52:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:52:48.741094 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e410febe-bbdb-4f0e-b5f7-10a88a0bce5b None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:48.742678 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-e410febe-bbdb-4f0e-b5f7-10a88a0bce5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:48.888463 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e410febe-bbdb-4f0e-b5f7-10a88a0bce5b None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:48.959836 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:52:49.000349 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84740ebe-205d-4fee-9b16-985b79d7fbb5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99655e1f9ca24472a86cd26b0dbd1fc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:49.000967 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84740ebe-205d-4fee-9b16-985b79d7fbb5 service nova] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:49.001251 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84740ebe-205d-4fee-9b16-985b79d7fbb5 service nova] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:49.001525 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84740ebe-205d-4fee-9b16-985b79d7fbb5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:49.003147 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-84740ebe-205d-4fee-9b16-985b79d7fbb5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:49.090966 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84740ebe-205d-4fee-9b16-985b79d7fbb5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:49.092611 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84740ebe-205d-4fee-9b16-985b79d7fbb5 service nova] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:49.110696 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 716/1416] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 09:52:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:52:50.005734 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76f7dc3-29ff-458d-831b-45637007441c None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:50.006287 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76f7dc3-29ff-458d-831b-45637007441c None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:50.006287 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76f7dc3-29ff-458d-831b-45637007441c None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:50.007639 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 701/1417] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 2 09:52:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:52:50.012417 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8aa94d-b16d-4c2b-843a-962d11d52fd4 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:50.012844 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8aa94d-b16d-4c2b-843a-962d11d52fd4 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:50.012974 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8aa94d-b16d-4c2b-843a-962d11d52fd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:50.089023 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-cd8aa94d-b16d-4c2b-843a-962d11d52fd4 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:52:50.090508 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-cd8aa94d-b16d-4c2b-843a-962d11d52fd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:50.117309 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:52:50.185280 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 717/1418] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 2 09:52:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:52:50.332481 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76f7dc3-29ff-458d-831b-45637007441c None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:50.334038 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-b76f7dc3-29ff-458d-831b-45637007441c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:50.476793 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76f7dc3-29ff-458d-831b-45637007441c None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:50.516830 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:52:50.589441 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdbd1c6-91f6-4119-a715-208c7e788a91 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99655e1f9ca24472a86cd26b0dbd1fc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:50.590316 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdbd1c6-91f6-4119-a715-208c7e788a91 service nova] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:50.590594 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdbd1c6-91f6-4119-a715-208c7e788a91 service nova] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:50.590867 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdbd1c6-91f6-4119-a715-208c7e788a91 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:50.592510 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-bbdbd1c6-91f6-4119-a715-208c7e788a91 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:50.679565 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdbd1c6-91f6-4119-a715-208c7e788a91 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:50.681212 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdbd1c6-91f6-4119-a715-208c7e788a91 service nova] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:50.699514 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 702/1419] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 09:52:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:52:51.576768 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ad8787-73ce-4115-9208-f7e203533275 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:51.577353 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ad8787-73ce-4115-9208-f7e203533275 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:51.577353 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ad8787-73ce-4115-9208-f7e203533275 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:51.578721 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 718/1420] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 2 09:52:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:52:51.583552 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04bd90a-d199-4121-95b6-feaaea294dc1 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:51.584007 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04bd90a-d199-4121-95b6-feaaea294dc1 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:51.584123 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04bd90a-d199-4121-95b6-feaaea294dc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:51.638121 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:52:51.660011 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-e04bd90a-d199-4121-95b6-feaaea294dc1 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:52:51.661279 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-e04bd90a-d199-4121-95b6-feaaea294dc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:51.757613 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 703/1421] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 2 09:52:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:52:51.907993 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ad8787-73ce-4115-9208-f7e203533275 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:51.909690 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-21ad8787-73ce-4115-9208-f7e203533275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:52.052848 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ad8787-73ce-4115-9208-f7e203533275 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:52.065331 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:52:52.163519 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f102215-c008-41ac-a9fe-08fe3de7ae2a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99655e1f9ca24472a86cd26b0dbd1fc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:52.164051 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f102215-c008-41ac-a9fe-08fe3de7ae2a service nova] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:52.164360 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f102215-c008-41ac-a9fe-08fe3de7ae2a service nova] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:52.164633 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f102215-c008-41ac-a9fe-08fe3de7ae2a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:52.166238 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-5f102215-c008-41ac-a9fe-08fe3de7ae2a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:52.254070 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f102215-c008-41ac-a9fe-08fe3de7ae2a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:52.255685 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f102215-c008-41ac-a9fe-08fe3de7ae2a service nova] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:52.274099 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 719/1422] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 09:52:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:52:53.130882 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9b1985-a544-409f-af51-ccbe1749255b None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:53.131447 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9b1985-a544-409f-af51-ccbe1749255b None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:53.131447 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9b1985-a544-409f-af51-ccbe1749255b None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:53.132849 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 704/1423] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 2 09:52:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 09:52:53.137814 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfee19e-fcf2-4fa3-ae2d-00beb9fd30e6 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:53.138288 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfee19e-fcf2-4fa3-ae2d-00beb9fd30e6 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:53.138393 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfee19e-fcf2-4fa3-ae2d-00beb9fd30e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:53.157717 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:52:53.214064 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-acfee19e-fcf2-4fa3-ae2d-00beb9fd30e6 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:52:53.215525 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-acfee19e-fcf2-4fa3-ae2d-00beb9fd30e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:53.312639 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 720/1424] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 2 09:52:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:52:53.463380 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9b1985-a544-409f-af51-ccbe1749255b None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:53.465101 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-6c9b1985-a544-409f-af51-ccbe1749255b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:53.574694 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:52:53.612659 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9b1985-a544-409f-af51-ccbe1749255b None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:53.727533 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c702ca71-9447-4e59-8e42-867c87eb34d3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99655e1f9ca24472a86cd26b0dbd1fc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:53.728074 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c702ca71-9447-4e59-8e42-867c87eb34d3 service nova] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:53.728389 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c702ca71-9447-4e59-8e42-867c87eb34d3 service nova] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:53.728668 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c702ca71-9447-4e59-8e42-867c87eb34d3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:53.730302 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-c702ca71-9447-4e59-8e42-867c87eb34d3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:53.805586 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:52:53.820020 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c702ca71-9447-4e59-8e42-867c87eb34d3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:53.821692 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c702ca71-9447-4e59-8e42-867c87eb34d3 service nova] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:53.840012 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 705/1425] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 09:52:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:52:54.691391 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faff01b-9816-4997-a25c-be1aa0a1635f None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:54.691991 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faff01b-9816-4997-a25c-be1aa0a1635f None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:54.691991 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faff01b-9816-4997-a25c-be1aa0a1635f None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:54.693319 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 721/1426] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 2 09:52:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:52:54.698327 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d9aad5-07de-443d-a454-1b12755790db None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:54.698747 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d9aad5-07de-443d-a454-1b12755790db None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:54.698862 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d9aad5-07de-443d-a454-1b12755790db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:54.773978 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-18d9aad5-07de-443d-a454-1b12755790db None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:52:54.775452 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-18d9aad5-07de-443d-a454-1b12755790db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:54.871204 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 706/1427] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 2 09:52:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:52:55.021197 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faff01b-9816-4997-a25c-be1aa0a1635f None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:55.022639 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-0faff01b-9816-4997-a25c-be1aa0a1635f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:55.106761 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:52:55.167931 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faff01b-9816-4997-a25c-be1aa0a1635f None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:55.276739 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35877484-6d19-4b07-91a4-70174f2fa8ee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99655e1f9ca24472a86cd26b0dbd1fc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:55.277273 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35877484-6d19-4b07-91a4-70174f2fa8ee service nova] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:55.277554 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35877484-6d19-4b07-91a4-70174f2fa8ee service nova] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:55.277826 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35877484-6d19-4b07-91a4-70174f2fa8ee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:55.279434 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-35877484-6d19-4b07-91a4-70174f2fa8ee service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:55.326490 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:52:55.368800 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35877484-6d19-4b07-91a4-70174f2fa8ee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:55.370411 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35877484-6d19-4b07-91a4-70174f2fa8ee service nova] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:55.388884 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 722/1428] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 09:52:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:52:56.201837 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bbcf29-30fd-469b-a710-f4d66090241d None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:56.202456 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bbcf29-30fd-469b-a710-f4d66090241d None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:56.202456 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bbcf29-30fd-469b-a710-f4d66090241d None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:56.204210 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 707/1429] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 2 09:52:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:52:56.209151 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cde869-5cd0-41b4-907e-39c7fbe5218d None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:56.209557 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cde869-5cd0-41b4-907e-39c7fbe5218d None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:56.209735 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cde869-5cd0-41b4-907e-39c7fbe5218d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:56.286312 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-a7cde869-5cd0-41b4-907e-39c7fbe5218d None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:52:56.287523 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-a7cde869-5cd0-41b4-907e-39c7fbe5218d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:56.385336 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 723/1430] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 2 09:52:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:52:56.536377 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bbcf29-30fd-469b-a710-f4d66090241d None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:56.537821 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-32bbcf29-30fd-469b-a710-f4d66090241d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:56.586771 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:52:56.682642 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bbcf29-30fd-469b-a710-f4d66090241d None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:56.793815 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10671438-95f8-442c-aea8-9c52b00c74b1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99655e1f9ca24472a86cd26b0dbd1fc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:56.794353 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10671438-95f8-442c-aea8-9c52b00c74b1 service nova] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:56.794632 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10671438-95f8-442c-aea8-9c52b00c74b1 service nova] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:56.794909 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10671438-95f8-442c-aea8-9c52b00c74b1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:56.797063 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-10671438-95f8-442c-aea8-9c52b00c74b1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:56.803832 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:52:56.885856 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10671438-95f8-442c-aea8-9c52b00c74b1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:56.887466 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10671438-95f8-442c-aea8-9c52b00c74b1 service nova] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:56.905727 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 708/1431] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 09:52:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:52:57.700339 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c085c145-e852-45f7-9799-8cb0ecb1f054 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:57.700949 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c085c145-e852-45f7-9799-8cb0ecb1f054 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:57.700949 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c085c145-e852-45f7-9799-8cb0ecb1f054 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:57.702230 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 724/1432] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 2 09:52:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:52:57.707255 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3005df8-a135-40f6-aba6-b3d403483ec3 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:57.707707 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3005df8-a135-40f6-aba6-b3d403483ec3 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:57.707816 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3005df8-a135-40f6-aba6-b3d403483ec3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:57.782624 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-b3005df8-a135-40f6-aba6-b3d403483ec3 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:52:57.783805 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-b3005df8-a135-40f6-aba6-b3d403483ec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:57.849726 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:52:57.879049 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 709/1433] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 2 09:52:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:52:58.029106 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c085c145-e852-45f7-9799-8cb0ecb1f054 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:58.030544 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-c085c145-e852-45f7-9799-8cb0ecb1f054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:58.048977 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:52:58.176977 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c085c145-e852-45f7-9799-8cb0ecb1f054 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:58.261750 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:52:58.287921 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0a999d-7358-4954-9550-525c72adda0d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99655e1f9ca24472a86cd26b0dbd1fc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:58.288498 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0a999d-7358-4954-9550-525c72adda0d service nova] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:58.288784 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0a999d-7358-4954-9550-525c72adda0d service nova] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:58.289060 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0a999d-7358-4954-9550-525c72adda0d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:58.290678 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-7e0a999d-7358-4954-9550-525c72adda0d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:58.377411 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0a999d-7358-4954-9550-525c72adda0d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:52:58.379010 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0a999d-7358-4954-9550-525c72adda0d service nova] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:52:58.397342 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 725/1434] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 09:52:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:52:59.192378 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff263b1-27fa-4442-9359-d30770b08d5e None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:59.192921 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff263b1-27fa-4442-9359-d30770b08d5e None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:59.192921 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff263b1-27fa-4442-9359-d30770b08d5e None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:59.194238 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 710/1435] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 2 09:52:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:52:59.199126 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8d5d38-c720-4f2d-8887-3c99ab4b4f2b None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:59.199509 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8d5d38-c720-4f2d-8887-3c99ab4b4f2b None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:59.199688 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8d5d38-c720-4f2d-8887-3c99ab4b4f2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:59.398779 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-ba8d5d38-c720-4f2d-8887-3c99ab4b4f2b None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:52:59.400017 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-ba8d5d38-c720-4f2d-8887-3c99ab4b4f2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:59.441728 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:52:59.496642 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 726/1436] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 2 09:52:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:52:59.648069 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff263b1-27fa-4442-9359-d30770b08d5e None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:52:59.649655 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-cff263b1-27fa-4442-9359-d30770b08d5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:52:59.792917 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff263b1-27fa-4442-9359-d30770b08d5e None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:52:59.847754 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:52:59.906761 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb3d851-c99f-45db-985b-ed5d325a8e3a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99655e1f9ca24472a86cd26b0dbd1fc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:52:59.907384 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb3d851-c99f-45db-985b-ed5d325a8e3a service nova] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:52:59.907671 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb3d851-c99f-45db-985b-ed5d325a8e3a service nova] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:52:59.907950 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb3d851-c99f-45db-985b-ed5d325a8e3a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:52:59.909615 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-dbb3d851-c99f-45db-985b-ed5d325a8e3a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:53:00.001375 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb3d851-c99f-45db-985b-ed5d325a8e3a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:53:00.002978 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb3d851-c99f-45db-985b-ed5d325a8e3a service nova] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:53:00.020988 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 711/1437] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 09:52:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:53:00.815716 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0754d30-462f-4680-ba40-2c50f4f354f5 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:00.816296 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0754d30-462f-4680-ba40-2c50f4f354f5 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:00.816296 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0754d30-462f-4680-ba40-2c50f4f354f5 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:00.817697 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 727/1438] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 2 09:53:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:53:00.822484 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c0e898-e106-4c28-8468-db51673882fe None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:00.822891 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c0e898-e106-4c28-8468-db51673882fe None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:00.823044 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c0e898-e106-4c28-8468-db51673882fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:00.899925 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-d9c0e898-e106-4c28-8468-db51673882fe None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:53:00.901166 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-d9c0e898-e106-4c28-8468-db51673882fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:53:00.903157 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:53:01.118572 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 712/1439] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 2 09:53:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:53:01.266473 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0754d30-462f-4680-ba40-2c50f4f354f5 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:53:01.267972 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-d0754d30-462f-4680-ba40-2c50f4f354f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:53:01.411326 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0754d30-462f-4680-ba40-2c50f4f354f5 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:53:01.438066 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:53:01.531815 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d282b9e-5e1f-49c3-8b01-da53baed06d1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99655e1f9ca24472a86cd26b0dbd1fc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:53:01.532462 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d282b9e-5e1f-49c3-8b01-da53baed06d1 service nova] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:01.532743 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d282b9e-5e1f-49c3-8b01-da53baed06d1 service nova] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:01.533015 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d282b9e-5e1f-49c3-8b01-da53baed06d1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:01.534635 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-3d282b9e-5e1f-49c3-8b01-da53baed06d1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:53:01.623263 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d282b9e-5e1f-49c3-8b01-da53baed06d1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:53:01.625407 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d282b9e-5e1f-49c3-8b01-da53baed06d1 service nova] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:53:01.643600 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 728/1440] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 09:53:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:53:03.533869 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f149c64-499a-42cf-a42d-ab5837fe181c None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:03.534429 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f149c64-499a-42cf-a42d-ab5837fe181c None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:03.534429 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f149c64-499a-42cf-a42d-ab5837fe181c None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:03.535761 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 713/1441] 127.0.0.1 () {54 vars in 916 bytes} [Tue Aug 2 09:53:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:53:03.549399 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6df431-00b8-4454-a6da-ea8af5bad1d9 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:03.549854 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6df431-00b8-4454-a6da-ea8af5bad1d9 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:03.549953 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6df431-00b8-4454-a6da-ea8af5bad1d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:03.591328 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:53:03.616759 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-3a6df431-00b8-4454-a6da-ea8af5bad1d9 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:53:03.617962 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-3a6df431-00b8-4454-a6da-ea8af5bad1d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:53:03.713794 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 729/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Aug 2 09:53:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:53:03.752970 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f149c64-499a-42cf-a42d-ab5837fe181c None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:53:03.754515 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-3f149c64-499a-42cf-a42d-ab5837fe181c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:53:03.855456 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f149c64-499a-42cf-a42d-ab5837fe181c None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:53:03.888640 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:53:03.930295 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576eb1b1-05ce-4fa1-a39b-717221c3075b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:53:03.930904 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576eb1b1-05ce-4fa1-a39b-717221c3075b service neutron] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:03.931183 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576eb1b1-05ce-4fa1-a39b-717221c3075b service neutron] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:03.931451 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576eb1b1-05ce-4fa1-a39b-717221c3075b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:03.933091 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-576eb1b1-05ce-4fa1-a39b-717221c3075b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:53:04.038653 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576eb1b1-05ce-4fa1-a39b-717221c3075b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:53:04.040074 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576eb1b1-05ce-4fa1-a39b-717221c3075b service neutron] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:53:04.058061 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 714/1443] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 09:53:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:53:06.187451 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445f9be5-3584-4afa-a5d9-6d065124183e None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:06.188010 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445f9be5-3584-4afa-a5d9-6d065124183e None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:06.188010 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445f9be5-3584-4afa-a5d9-6d065124183e None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:06.189287 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 730/1444] 127.0.0.1 () {54 vars in 916 bytes} [Tue Aug 2 09:53:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:53:06.194371 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d8526c-17b0-4535-befd-da4e02a9de2c None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:06.194768 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d8526c-17b0-4535-befd-da4e02a9de2c None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:06.194914 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d8526c-17b0-4535-befd-da4e02a9de2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:06.261531 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-56d8526c-17b0-4535-befd-da4e02a9de2c None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:53:06.262737 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-56d8526c-17b0-4535-befd-da4e02a9de2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:53:06.275927 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:53:06.357710 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 715/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Aug 2 09:53:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:53:06.396618 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445f9be5-3584-4afa-a5d9-6d065124183e None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:53:06.398155 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-445f9be5-3584-4afa-a5d9-6d065124183e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:53:06.500636 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445f9be5-3584-4afa-a5d9-6d065124183e None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:53:06.512478 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:53:06.572791 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0479d506-a93a-4105-a45d-c112127539cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:53:06.573401 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0479d506-a93a-4105-a45d-c112127539cd service neutron] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:06.573686 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0479d506-a93a-4105-a45d-c112127539cd service neutron] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:06.573961 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0479d506-a93a-4105-a45d-c112127539cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:06.575574 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-0479d506-a93a-4105-a45d-c112127539cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:53:06.682778 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0479d506-a93a-4105-a45d-c112127539cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:53:06.684226 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0479d506-a93a-4105-a45d-c112127539cd service neutron] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:53:06.703462 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 731/1446] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 09:53:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:53:09.005003 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95dbf24-8fe8-4e29-b0f4-26f1077129b2 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:09.005579 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95dbf24-8fe8-4e29-b0f4-26f1077129b2 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:09.005579 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95dbf24-8fe8-4e29-b0f4-26f1077129b2 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:09.006916 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 716/1447] 127.0.0.1 () {54 vars in 916 bytes} [Tue Aug 2 09:53:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:53:09.012452 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b5796d-3d9e-46cd-a99c-cc7d1483aa15 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:09.012889 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b5796d-3d9e-46cd-a99c-cc7d1483aa15 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:09.013015 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b5796d-3d9e-46cd-a99c-cc7d1483aa15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:09.066486 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:53:09.085632 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-a1b5796d-3d9e-46cd-a99c-cc7d1483aa15 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:53:09.086850 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-a1b5796d-3d9e-46cd-a99c-cc7d1483aa15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:53:09.185036 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 732/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Aug 2 09:53:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:53:09.224692 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95dbf24-8fe8-4e29-b0f4-26f1077129b2 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:53:09.226228 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-c95dbf24-8fe8-4e29-b0f4-26f1077129b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:53:09.328990 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95dbf24-8fe8-4e29-b0f4-26f1077129b2 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:53:09.368933 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:53:09.403716 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226f8e4a-f01d-4241-bfa0-15b4a3b15efd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:53:09.404351 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226f8e4a-f01d-4241-bfa0-15b4a3b15efd service neutron] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:09.404629 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226f8e4a-f01d-4241-bfa0-15b4a3b15efd service neutron] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:09.404896 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226f8e4a-f01d-4241-bfa0-15b4a3b15efd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:09.407113 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-226f8e4a-f01d-4241-bfa0-15b4a3b15efd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:53:09.513842 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226f8e4a-f01d-4241-bfa0-15b4a3b15efd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:53:09.515709 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226f8e4a-f01d-4241-bfa0-15b4a3b15efd service neutron] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:53:09.533755 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 717/1449] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 09:53:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:53:11.640821 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a507bd2-8c40-4b88-a320-4ac67fd17d21 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:11.641368 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a507bd2-8c40-4b88-a320-4ac67fd17d21 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:11.641368 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a507bd2-8c40-4b88-a320-4ac67fd17d21 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:11.642699 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 733/1450] 127.0.0.1 () {54 vars in 916 bytes} [Tue Aug 2 09:53:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 09:53:11.648121 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88bb68d-159b-48c1-b7e3-8dcd120ecefa None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:11.648607 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88bb68d-159b-48c1-b7e3-8dcd120ecefa None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:11.648714 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88bb68d-159b-48c1-b7e3-8dcd120ecefa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:11.716265 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-e88bb68d-159b-48c1-b7e3-8dcd120ecefa None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:53:11.717492 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-e88bb68d-159b-48c1-b7e3-8dcd120ecefa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:53:11.738112 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:53:11.814608 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 718/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Aug 2 09:53:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:53:11.854292 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a507bd2-8c40-4b88-a320-4ac67fd17d21 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:53:11.855891 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-8a507bd2-8c40-4b88-a320-4ac67fd17d21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:53:11.959009 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a507bd2-8c40-4b88-a320-4ac67fd17d21 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:53:11.979152 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:53:12.033141 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7534d565-8e43-4047-a36a-e8a571398c03 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:53:12.033760 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7534d565-8e43-4047-a36a-e8a571398c03 service neutron] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:12.034046 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7534d565-8e43-4047-a36a-e8a571398c03 service neutron] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:12.034321 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7534d565-8e43-4047-a36a-e8a571398c03 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:12.035949 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-7534d565-8e43-4047-a36a-e8a571398c03 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:53:12.143880 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7534d565-8e43-4047-a36a-e8a571398c03 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:53:12.145354 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7534d565-8e43-4047-a36a-e8a571398c03 service neutron] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:53:12.163615 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 734/1452] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 09:53:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:53:14.341449 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145276e3-16b8-4cd3-ac6e-3d05f17ac8d8 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:14.341984 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145276e3-16b8-4cd3-ac6e-3d05f17ac8d8 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:14.341984 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145276e3-16b8-4cd3-ac6e-3d05f17ac8d8 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:14.343362 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 719/1453] 127.0.0.1 () {54 vars in 916 bytes} [Tue Aug 2 09:53:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:53:14.348762 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946afd7b-0ad8-4219-8e8e-ff6410746e5c None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:14.349194 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946afd7b-0ad8-4219-8e8e-ff6410746e5c None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:14.349302 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946afd7b-0ad8-4219-8e8e-ff6410746e5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:14.414824 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:53:14.417639 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-946afd7b-0ad8-4219-8e8e-ff6410746e5c None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:53:14.419159 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-946afd7b-0ad8-4219-8e8e-ff6410746e5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:53:14.515760 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 735/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Aug 2 09:53:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:53:14.554883 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145276e3-16b8-4cd3-ac6e-3d05f17ac8d8 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:53:14.556465 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-145276e3-16b8-4cd3-ac6e-3d05f17ac8d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:53:14.659817 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145276e3-16b8-4cd3-ac6e-3d05f17ac8d8 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:53:14.707936 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:53:14.732066 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481865b1-8017-4cc7-8b45-6078c44c2f25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:53:14.732710 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481865b1-8017-4cc7-8b45-6078c44c2f25 service neutron] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:14.732993 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481865b1-8017-4cc7-8b45-6078c44c2f25 service neutron] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:14.733775 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481865b1-8017-4cc7-8b45-6078c44c2f25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:14.735389 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-481865b1-8017-4cc7-8b45-6078c44c2f25 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:53:14.841016 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481865b1-8017-4cc7-8b45-6078c44c2f25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:53:14.842459 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481865b1-8017-4cc7-8b45-6078c44c2f25 service neutron] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:53:14.861337 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 720/1455] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 09:53:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:53:16.897844 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e920ed01-2636-4288-a83c-4aed18a20d90 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:16.898303 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e920ed01-2636-4288-a83c-4aed18a20d90 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:16.898443 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e920ed01-2636-4288-a83c-4aed18a20d90 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:16.899801 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 736/1456] 127.0.0.1 () {54 vars in 916 bytes} [Tue Aug 2 09:53:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:53:16.905763 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1aeaeae-fe88-4ae9-bb86-83e1780ece9d None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:16.906186 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1aeaeae-fe88-4ae9-bb86-83e1780ece9d None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:16.906313 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1aeaeae-fe88-4ae9-bb86-83e1780ece9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:16.974541 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-a1aeaeae-fe88-4ae9-bb86-83e1780ece9d None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:53:16.975667 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-a1aeaeae-fe88-4ae9-bb86-83e1780ece9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:53:17.004434 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:53:17.070963 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 721/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Aug 2 09:53:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:53:17.108671 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e920ed01-2636-4288-a83c-4aed18a20d90 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:53:17.110238 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-e920ed01-2636-4288-a83c-4aed18a20d90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:53:17.210902 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e920ed01-2636-4288-a83c-4aed18a20d90 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:53:17.241243 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:53:17.288794 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d2d6d8-6783-4798-8b85-ad83f90f5a9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:53:17.289406 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d2d6d8-6783-4798-8b85-ad83f90f5a9d service neutron] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:17.289690 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d2d6d8-6783-4798-8b85-ad83f90f5a9d service neutron] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:17.289971 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d2d6d8-6783-4798-8b85-ad83f90f5a9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:17.291573 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-44d2d6d8-6783-4798-8b85-ad83f90f5a9d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:53:17.400975 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d2d6d8-6783-4798-8b85-ad83f90f5a9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:53:17.402414 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d2d6d8-6783-4798-8b85-ad83f90f5a9d service neutron] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:53:17.419786 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 737/1458] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 09:53:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:53:19.552910 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18816ffd-35cf-4493-90e7-b7b4eb325664 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:19.553500 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18816ffd-35cf-4493-90e7-b7b4eb325664 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:19.553500 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18816ffd-35cf-4493-90e7-b7b4eb325664 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:19.554808 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 722/1459] 127.0.0.1 () {54 vars in 916 bytes} [Tue Aug 2 09:53:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:53:19.559946 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eff982a-70de-4768-88e8-f471a5c8bb64 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:19.560396 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eff982a-70de-4768-88e8-f471a5c8bb64 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:19.560562 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eff982a-70de-4768-88e8-f471a5c8bb64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:19.627796 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-0eff982a-70de-4768-88e8-f471a5c8bb64 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:53:19.629056 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-0eff982a-70de-4768-88e8-f471a5c8bb64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:53:19.637283 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:53:19.724864 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 738/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Aug 2 09:53:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:53:19.764032 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18816ffd-35cf-4493-90e7-b7b4eb325664 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:53:19.766285 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-18816ffd-35cf-4493-90e7-b7b4eb325664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:53:19.869485 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18816ffd-35cf-4493-90e7-b7b4eb325664 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:53:19.925279 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:53:19.943586 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda5672d-8610-4ec9-8720-dd7d4bd5a381 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:53:19.944207 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda5672d-8610-4ec9-8720-dd7d4bd5a381 service neutron] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:19.944519 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda5672d-8610-4ec9-8720-dd7d4bd5a381 service neutron] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:19.944793 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda5672d-8610-4ec9-8720-dd7d4bd5a381 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:19.946417 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-bda5672d-8610-4ec9-8720-dd7d4bd5a381 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:53:20.053633 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda5672d-8610-4ec9-8720-dd7d4bd5a381 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:53:20.055094 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda5672d-8610-4ec9-8720-dd7d4bd5a381 service neutron] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:53:20.073488 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 723/1461] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 09:53:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:53:22.105950 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab689fc8-c43b-4fe7-a507-759353fdc5f9 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:22.106512 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab689fc8-c43b-4fe7-a507-759353fdc5f9 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:22.106512 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab689fc8-c43b-4fe7-a507-759353fdc5f9 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:22.107873 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 739/1462] 127.0.0.1 () {54 vars in 916 bytes} [Tue Aug 2 09:53:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 09:53:22.113585 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc09dc7-1958-4941-a8a8-be46c6f8fecb None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:22.114021 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc09dc7-1958-4941-a8a8-be46c6f8fecb None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:22.114148 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc09dc7-1958-4941-a8a8-be46c6f8fecb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:22.179009 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-ddc09dc7-1958-4941-a8a8-be46c6f8fecb None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:53:22.180141 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-ddc09dc7-1958-4941-a8a8-be46c6f8fecb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:53:22.218255 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:53:22.276043 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 724/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Aug 2 09:53:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:53:22.313884 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab689fc8-c43b-4fe7-a507-759353fdc5f9 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:53:22.315422 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-ab689fc8-c43b-4fe7-a507-759353fdc5f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:53:22.419277 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab689fc8-c43b-4fe7-a507-759353fdc5f9 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:53:22.456073 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:53:22.491469 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d316b0-0280-470e-9485-c807ca043f5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:53:22.492073 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d316b0-0280-470e-9485-c807ca043f5a service neutron] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:22.492389 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d316b0-0280-470e-9485-c807ca043f5a service neutron] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:22.492673 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d316b0-0280-470e-9485-c807ca043f5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:22.494294 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-c5d316b0-0280-470e-9485-c807ca043f5a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:53:22.601941 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d316b0-0280-470e-9485-c807ca043f5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:53:22.603383 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d316b0-0280-470e-9485-c807ca043f5a service neutron] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:53:22.620977 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 740/1464] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 09:53:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:53:24.650642 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab471e2b-8d37-4d22-984c-494a8869b0e7 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:24.651219 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab471e2b-8d37-4d22-984c-494a8869b0e7 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:24.651219 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab471e2b-8d37-4d22-984c-494a8869b0e7 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:24.652584 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 725/1465] 127.0.0.1 () {54 vars in 916 bytes} [Tue Aug 2 09:53:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:53:24.657816 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4812c5-2533-4673-8bc7-0d745aaeabcd None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:24.658244 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4812c5-2533-4673-8bc7-0d745aaeabcd None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:24.658375 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4812c5-2533-4673-8bc7-0d745aaeabcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:24.724871 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-7d4812c5-2533-4673-8bc7-0d745aaeabcd None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:53:24.726079 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-7d4812c5-2533-4673-8bc7-0d745aaeabcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:53:24.744166 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:53:24.822097 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 741/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Aug 2 09:53:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:53:24.859440 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab471e2b-8d37-4d22-984c-494a8869b0e7 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:53:24.861007 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-ab471e2b-8d37-4d22-984c-494a8869b0e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:53:24.961367 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab471e2b-8d37-4d22-984c-494a8869b0e7 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:53:25.026001 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:53:25.033445 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b348b94-79ae-4b43-be93-3769e546e2cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:53:25.034061 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b348b94-79ae-4b43-be93-3769e546e2cd service neutron] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:25.034346 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b348b94-79ae-4b43-be93-3769e546e2cd service neutron] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:25.034627 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b348b94-79ae-4b43-be93-3769e546e2cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:25.036276 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-7b348b94-79ae-4b43-be93-3769e546e2cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:53:25.144105 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b348b94-79ae-4b43-be93-3769e546e2cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:53:25.145570 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b348b94-79ae-4b43-be93-3769e546e2cd service neutron] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:53:25.163648 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 726/1467] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 09:53:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:53:27.206015 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc104b0-4206-498d-9172-1e783fa2653b None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:27.206621 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc104b0-4206-498d-9172-1e783fa2653b None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:27.206621 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc104b0-4206-498d-9172-1e783fa2653b None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:27.207925 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 742/1468] 127.0.0.1 () {54 vars in 916 bytes} [Tue Aug 2 09:53:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 09:53:27.213268 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2b0791-95b3-46cc-98ca-6bb3cd504bf0 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:27.213962 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2b0791-95b3-46cc-98ca-6bb3cd504bf0 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:27.214078 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2b0791-95b3-46cc-98ca-6bb3cd504bf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:27.279841 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-2c2b0791-95b3-46cc-98ca-6bb3cd504bf0 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:53:27.281092 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-2c2b0791-95b3-46cc-98ca-6bb3cd504bf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:53:27.330876 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:53:27.379911 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 727/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Aug 2 09:53:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:53:27.416954 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc104b0-4206-498d-9172-1e783fa2653b None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:53:27.418484 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-bdc104b0-4206-498d-9172-1e783fa2653b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:53:27.520582 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc104b0-4206-498d-9172-1e783fa2653b None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:53:27.568643 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:53:27.594535 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c1d874-ef26-49d3-8289-c250ec4ae68c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:53:27.595142 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c1d874-ef26-49d3-8289-c250ec4ae68c service neutron] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:27.595423 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c1d874-ef26-49d3-8289-c250ec4ae68c service neutron] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:27.595693 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c1d874-ef26-49d3-8289-c250ec4ae68c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:27.597336 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-41c1d874-ef26-49d3-8289-c250ec4ae68c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:53:27.703393 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c1d874-ef26-49d3-8289-c250ec4ae68c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:53:27.704868 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c1d874-ef26-49d3-8289-c250ec4ae68c service neutron] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:53:27.723084 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 743/1470] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 09:53:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:53:37.378088 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=22 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:53:37.720910 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=66 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:53:47.101295 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d35910-a90f-45bf-adae-33f781be0fa9 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:47.101832 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d35910-a90f-45bf-adae-33f781be0fa9 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:47.101832 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d35910-a90f-45bf-adae-33f781be0fa9 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:47.103248 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 728/1471] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:53:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:53:47.108599 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949e67eb-4935-4922-9c96-28bf388eb9d5 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:47.109022 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949e67eb-4935-4922-9c96-28bf388eb9d5 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:47.109246 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949e67eb-4935-4922-9c96-28bf388eb9d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:47.175171 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-949e67eb-4935-4922-9c96-28bf388eb9d5 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:53:47.176412 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-949e67eb-4935-4922-9c96-28bf388eb9d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:53:47.271801 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 744/1472] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:53:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:53:47.356809 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cc2e43-4e25-47ec-b2e3-c3d0fc2cbc8b None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:47.357796 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cc2e43-4e25-47ec-b2e3-c3d0fc2cbc8b None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:47.358074 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cc2e43-4e25-47ec-b2e3-c3d0fc2cbc8b None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:47.359409 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 729/1473] 10.222.0.64 () {54 vars in 966 bytes} [Tue Aug 2 09:53:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:53:47.364223 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9ce8ef-8228-4932-881f-f1a426171334 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:47.364554 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9ce8ef-8228-4932-881f-f1a426171334 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:47.364836 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9ce8ef-8228-4932-881f-f1a426171334 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:47.439327 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-ce9ce8ef-8228-4932-881f-f1a426171334 None None] MFA Rules not processed for user `04fb1dbe260a4721aaa0bd0493260d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:53:47.440566 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-ce9ce8ef-8228-4932-881f-f1a426171334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:53:47.460595 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:53:47.516887 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 745/1474] 10.222.0.64 () {58 vars in 1063 bytes} [Tue Aug 2 09:53:47 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:53:47.524664 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cc2e43-4e25-47ec-b2e3-c3d0fc2cbc8b None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:53:47.526207 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-f3cc2e43-4e25-47ec-b2e3-c3d0fc2cbc8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:53:47.627972 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cc2e43-4e25-47ec-b2e3-c3d0fc2cbc8b None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:53:47.698453 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08046e87-b116-40c5-9d9c-9aa2a22bbd8e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04fb1dbe260a4721aaa0bd0493260d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:53:47.699051 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08046e87-b116-40c5-9d9c-9aa2a22bbd8e service glance] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:47.699331 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08046e87-b116-40c5-9d9c-9aa2a22bbd8e service glance] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:47.699602 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08046e87-b116-40c5-9d9c-9aa2a22bbd8e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:47.701510 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-08046e87-b116-40c5-9d9c-9aa2a22bbd8e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:53:47.754288 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:53:47.808709 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08046e87-b116-40c5-9d9c-9aa2a22bbd8e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:53:47.810672 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08046e87-b116-40c5-9d9c-9aa2a22bbd8e service glance] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:53:47.829018 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 730/1475] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 09:53:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:53:49.916728 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecd98a4-e9a3-4cc2-ae73-4a408cb3963f None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:49.917192 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecd98a4-e9a3-4cc2-ae73-4a408cb3963f None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:49.917296 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecd98a4-e9a3-4cc2-ae73-4a408cb3963f None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:49.918637 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 746/1476] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:53:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:53:49.924314 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af029646-b9c3-40a3-9428-906c58c37660 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:49.924761 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af029646-b9c3-40a3-9428-906c58c37660 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:49.924905 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af029646-b9c3-40a3-9428-906c58c37660 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:49.990086 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-af029646-b9c3-40a3-9428-906c58c37660 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:53:49.991280 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-af029646-b9c3-40a3-9428-906c58c37660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:53:50.084593 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 731/1477] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:53:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:53:50.108182 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecd98a4-e9a3-4cc2-ae73-4a408cb3963f None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:53:50.109773 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-aecd98a4-e9a3-4cc2-ae73-4a408cb3963f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:53:50.234604 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecd98a4-e9a3-4cc2-ae73-4a408cb3963f None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:53:50.283540 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:53:50.327535 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa1f5bc-4d50-40a4-8f38-eb3aa7eacbd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:53:50.328150 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa1f5bc-4d50-40a4-8f38-eb3aa7eacbd0 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:50.328462 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa1f5bc-4d50-40a4-8f38-eb3aa7eacbd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:50.328737 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa1f5bc-4d50-40a4-8f38-eb3aa7eacbd0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:50.329432 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa1f5bc-4d50-40a4-8f38-eb3aa7eacbd0 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:53:50.331166 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: WARNING py.warnings [None req-daa1f5bc-4d50-40a4-8f38-eb3aa7eacbd0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 02 09:53:50.331166 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: warnings.warn(msg) Aug 02 09:53:50.331166 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: Aug 02 09:53:50.332446 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa1f5bc-4d50-40a4-8f38-eb3aa7eacbd0 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:53:50.351440 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 747/1478] 10.222.0.64 () {56 vars in 1108 bytes} [Tue Aug 2 09:53:50 2022] GET /identity/v3/projects => generated 2553 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:53:52.366886 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c10a848-fe7c-46cb-9127-5942aa20f40e None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:52.367442 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c10a848-fe7c-46cb-9127-5942aa20f40e None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:52.367442 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c10a848-fe7c-46cb-9127-5942aa20f40e None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:52.368867 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 732/1479] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:53:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:53:52.374502 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12a169b-e8df-474a-94e2-6a711055ef81 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:52.374910 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12a169b-e8df-474a-94e2-6a711055ef81 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:52.375069 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12a169b-e8df-474a-94e2-6a711055ef81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:52.441053 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-d12a169b-e8df-474a-94e2-6a711055ef81 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:53:52.442297 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-d12a169b-e8df-474a-94e2-6a711055ef81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:53:52.538225 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 748/1480] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:53:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:53:52.581071 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c10a848-fe7c-46cb-9127-5942aa20f40e None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:53:52.582626 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-7c10a848-fe7c-46cb-9127-5942aa20f40e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:53:52.593345 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:53:52.728033 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c10a848-fe7c-46cb-9127-5942aa20f40e None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:53:52.806716 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:53:52.839966 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f25c7b-c84f-43c4-828c-9cc2980c06df service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99655e1f9ca24472a86cd26b0dbd1fc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:53:52.840614 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f25c7b-c84f-43c4-828c-9cc2980c06df service nova] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:52.840894 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f25c7b-c84f-43c4-828c-9cc2980c06df service nova] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:52.841729 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f25c7b-c84f-43c4-828c-9cc2980c06df service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:52.843354 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-80f25c7b-c84f-43c4-828c-9cc2980c06df service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:53:52.931646 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f25c7b-c84f-43c4-828c-9cc2980c06df service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:53:52.933359 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f25c7b-c84f-43c4-828c-9cc2980c06df service nova] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:53:52.952052 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 733/1481] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 09:53:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:53:55.116555 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3521ce-6732-4054-816d-71c9eed14461 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:55.117118 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3521ce-6732-4054-816d-71c9eed14461 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:55.117118 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3521ce-6732-4054-816d-71c9eed14461 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:55.118470 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 749/1482] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:53:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:53:55.124404 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59d1ad2-b81f-4f6a-aaa7-86cc73508aee None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:55.124838 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59d1ad2-b81f-4f6a-aaa7-86cc73508aee None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:55.124942 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59d1ad2-b81f-4f6a-aaa7-86cc73508aee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:55.191362 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-f59d1ad2-b81f-4f6a-aaa7-86cc73508aee None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:53:55.192597 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-f59d1ad2-b81f-4f6a-aaa7-86cc73508aee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:53:55.238902 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:53:55.289823 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 734/1483] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:53:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:53:55.326541 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3521ce-6732-4054-816d-71c9eed14461 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:53:55.328100 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-ba3521ce-6732-4054-816d-71c9eed14461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:53:55.352307 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:53:55.432515 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3521ce-6732-4054-816d-71c9eed14461 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:53:55.505845 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2a1488-21ee-440d-bfc2-004dea013af2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:53:55.506485 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2a1488-21ee-440d-bfc2-004dea013af2 service neutron] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:55.506772 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2a1488-21ee-440d-bfc2-004dea013af2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:55.507081 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2a1488-21ee-440d-bfc2-004dea013af2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:55.508738 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-5d2a1488-21ee-440d-bfc2-004dea013af2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:53:55.581160 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:53:55.615505 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2a1488-21ee-440d-bfc2-004dea013af2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:53:55.616971 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2a1488-21ee-440d-bfc2-004dea013af2 service neutron] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:53:55.634993 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 750/1484] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 09:53:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:53:57.697880 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbbe700-bd08-47b6-bf18-a71dac7fa7e5 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:57.698570 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbbe700-bd08-47b6-bf18-a71dac7fa7e5 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:57.698570 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbbe700-bd08-47b6-bf18-a71dac7fa7e5 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:57.699783 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 735/1485] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:53:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:53:57.705738 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29144df9-09c3-4e70-8915-c60f4212a551 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:57.706150 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29144df9-09c3-4e70-8915-c60f4212a551 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:57.706297 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29144df9-09c3-4e70-8915-c60f4212a551 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:57.772490 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-29144df9-09c3-4e70-8915-c60f4212a551 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:53:57.773704 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-29144df9-09c3-4e70-8915-c60f4212a551 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:53:57.869173 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 751/1486] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:53:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:53:57.909637 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbbe700-bd08-47b6-bf18-a71dac7fa7e5 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:53:57.911184 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-3cbbe700-bd08-47b6-bf18-a71dac7fa7e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:53:58.013019 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbbe700-bd08-47b6-bf18-a71dac7fa7e5 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:53:58.085867 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:53:58.089025 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7806e676-558a-4bab-9c80-05a9e502a3ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:53:58.089878 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7806e676-558a-4bab-9c80-05a9e502a3ca service neutron] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:53:58.090325 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7806e676-558a-4bab-9c80-05a9e502a3ca service neutron] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:53:58.090805 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7806e676-558a-4bab-9c80-05a9e502a3ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:53:58.092486 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-7806e676-558a-4bab-9c80-05a9e502a3ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:53:58.199953 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7806e676-558a-4bab-9c80-05a9e502a3ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:53:58.201448 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7806e676-558a-4bab-9c80-05a9e502a3ca service neutron] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:53:58.219602 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 736/1487] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 09:53:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:54:00.321145 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b84a3b-b6dd-4140-882a-d2f1dd325092 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:54:00.321740 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b84a3b-b6dd-4140-882a-d2f1dd325092 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:54:00.321740 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b84a3b-b6dd-4140-882a-d2f1dd325092 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:54:00.323067 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 752/1488] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:54:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:54:00.329165 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3640971-54c2-4986-98d3-0fa06bd09aff None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:54:00.329630 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3640971-54c2-4986-98d3-0fa06bd09aff None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:54:00.329735 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3640971-54c2-4986-98d3-0fa06bd09aff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:54:00.396498 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-a3640971-54c2-4986-98d3-0fa06bd09aff None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:54:00.397716 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-a3640971-54c2-4986-98d3-0fa06bd09aff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:54:00.449566 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:54:00.493493 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 737/1489] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:54:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:54:00.534834 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b84a3b-b6dd-4140-882a-d2f1dd325092 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:54:00.536448 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-12b84a3b-b6dd-4140-882a-d2f1dd325092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:54:00.574928 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:54:00.643466 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b84a3b-b6dd-4140-882a-d2f1dd325092 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:54:00.714192 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e71572-c249-4b41-a2f2-1ce1ab63eb54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:54:00.714823 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e71572-c249-4b41-a2f2-1ce1ab63eb54 service neutron] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:54:00.715102 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e71572-c249-4b41-a2f2-1ce1ab63eb54 service neutron] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:54:00.715372 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e71572-c249-4b41-a2f2-1ce1ab63eb54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:54:00.717682 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-c9e71572-c249-4b41-a2f2-1ce1ab63eb54 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:54:00.797840 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:54:00.828999 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e71572-c249-4b41-a2f2-1ce1ab63eb54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:54:00.830382 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e71572-c249-4b41-a2f2-1ce1ab63eb54 service neutron] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:54:00.848820 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 753/1490] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 09:54:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:54:03.037559 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feaa7dc-9a38-4b1a-8d3e-8553f09f9503 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:54:03.038095 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feaa7dc-9a38-4b1a-8d3e-8553f09f9503 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:54:03.038095 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feaa7dc-9a38-4b1a-8d3e-8553f09f9503 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:54:03.039374 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 738/1491] 10.222.0.64 () {54 vars in 926 bytes} [Tue Aug 2 09:54:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:54:03.045242 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60928851-ef67-428e-8d46-c4841b428b6d None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:54:03.045672 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60928851-ef67-428e-8d46-c4841b428b6d None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:54:03.045810 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60928851-ef67-428e-8d46-c4841b428b6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:54:03.113174 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-60928851-ef67-428e-8d46-c4841b428b6d None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:54:03.114379 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-60928851-ef67-428e-8d46-c4841b428b6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:54:03.209987 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 754/1492] 10.222.0.64 () {58 vars in 1023 bytes} [Tue Aug 2 09:54:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:54:03.249100 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feaa7dc-9a38-4b1a-8d3e-8553f09f9503 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:54:03.250627 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-2feaa7dc-9a38-4b1a-8d3e-8553f09f9503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:54:03.354101 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feaa7dc-9a38-4b1a-8d3e-8553f09f9503 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:54:03.426587 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9933ff04-3179-4c46-a648-5f0ea02afbc2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:54:03.427189 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9933ff04-3179-4c46-a648-5f0ea02afbc2 service neutron] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:54:03.427468 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9933ff04-3179-4c46-a648-5f0ea02afbc2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:54:03.427738 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9933ff04-3179-4c46-a648-5f0ea02afbc2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:54:03.429391 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-9933ff04-3179-4c46-a648-5f0ea02afbc2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:54:03.438878 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:54:03.535576 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9933ff04-3179-4c46-a648-5f0ea02afbc2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:54:03.537105 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9933ff04-3179-4c46-a648-5f0ea02afbc2 service neutron] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:54:03.555253 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 739/1493] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 09:54:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:54:13.208151 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=82 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:54:13.553075 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=50 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:55:30.255294 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c89bb3c-3831-4057-aac7-fb17e7468b99 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:55:30.255869 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c89bb3c-3831-4057-aac7-fb17e7468b99 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:55:30.255869 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c89bb3c-3831-4057-aac7-fb17e7468b99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:55:30.335620 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-4c89bb3c-3831-4057-aac7-fb17e7468b99 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:55:30.337177 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-4c89bb3c-3831-4057-aac7-fb17e7468b99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:55:30.432714 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 755/1494] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 09:55:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:55:30.440904 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9450ebb1-7c74-4f2a-a28c-5131a3b8ff62 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:55:30.441477 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9450ebb1-7c74-4f2a-a28c-5131a3b8ff62 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:55:30.441477 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9450ebb1-7c74-4f2a-a28c-5131a3b8ff62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:55:30.517168 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-9450ebb1-7c74-4f2a-a28c-5131a3b8ff62 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:55:30.518306 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-9450ebb1-7c74-4f2a-a28c-5131a3b8ff62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:55:30.613106 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 740/1495] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 09:55:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:55:30.617686 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c89bb3c-3831-4057-aac7-fb17e7468b99 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:55:30.619551 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-4c89bb3c-3831-4057-aac7-fb17e7468b99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:55:30.631095 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c89bb3c-3831-4057-aac7-fb17e7468b99 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:55:30.690544 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:55:30.725258 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886d6dc2-931f-4785-8eb7-693fcd099a60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:55:30.725865 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886d6dc2-931f-4785-8eb7-693fcd099a60 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:55:30.726148 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886d6dc2-931f-4785-8eb7-693fcd099a60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:55:30.726418 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886d6dc2-931f-4785-8eb7-693fcd099a60 admin admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:55:30.727100 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886d6dc2-931f-4785-8eb7-693fcd099a60 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:55:30.727996 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886d6dc2-931f-4785-8eb7-693fcd099a60 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:55:30.729038 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: WARNING py.warnings [None req-886d6dc2-931f-4785-8eb7-693fcd099a60 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 02 09:55:30.729038 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: warnings.warn(msg) Aug 02 09:55:30.729038 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: Aug 02 09:55:30.729928 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886d6dc2-931f-4785-8eb7-693fcd099a60 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:55:30.740832 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 756/1496] 10.222.0.64 () {58 vars in 1153 bytes} [Tue Aug 2 09:55:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:55:30.749111 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9450ebb1-7c74-4f2a-a28c-5131a3b8ff62 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:55:30.758296 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9450ebb1-7c74-4f2a-a28c-5131a3b8ff62 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:55:30.819590 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:55:30.853895 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cd72bb-f14a-432b-a025-6bef2607e15b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:55:30.854505 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cd72bb-f14a-432b-a025-6bef2607e15b admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:55:30.854784 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cd72bb-f14a-432b-a025-6bef2607e15b admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:55:30.855055 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cd72bb-f14a-432b-a025-6bef2607e15b admin admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:55:30.855709 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cd72bb-f14a-432b-a025-6bef2607e15b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:55:30.857466 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: WARNING py.warnings [None req-17cd72bb-f14a-432b-a025-6bef2607e15b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 02 09:55:30.857466 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: warnings.warn(msg) Aug 02 09:55:30.857466 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: Aug 02 09:55:30.858643 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cd72bb-f14a-432b-a025-6bef2607e15b admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:55:30.912662 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 741/1497] 10.222.0.64 () {60 vars in 1153 bytes} [Tue Aug 2 09:55:30 2022] POST /identity/v3/projects => generated 377 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:55:30.917381 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886d6dc2-931f-4785-8eb7-693fcd099a60 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:55:30.918911 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-886d6dc2-931f-4785-8eb7-693fcd099a60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:55:30.951447 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886d6dc2-931f-4785-8eb7-693fcd099a60 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:55:31.044236 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9d89e5-aeb6-45c9-9d84-b699fcde4cb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:55:31.044823 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9d89e5-aeb6-45c9-9d84-b699fcde4cb7 admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:55:31.045086 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9d89e5-aeb6-45c9-9d84-b699fcde4cb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:55:31.045359 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9d89e5-aeb6-45c9-9d84-b699fcde4cb7 admin admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:55:31.046024 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9d89e5-aeb6-45c9-9d84-b699fcde4cb7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:55:31.047035 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: WARNING py.warnings [None req-9a9d89e5-aeb6-45c9-9d84-b699fcde4cb7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 02 09:55:31.047035 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: warnings.warn(msg) Aug 02 09:55:31.047035 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: Aug 02 09:55:31.048231 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9d89e5-aeb6-45c9-9d84-b699fcde4cb7 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:55:31.089893 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 757/1498] 10.222.0.64 () {60 vars in 1144 bytes} [Tue Aug 2 09:55:30 2022] POST /identity/v3/users => generated 289 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:55:31.094173 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cd72bb-f14a-432b-a025-6bef2607e15b admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:55:31.095752 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-17cd72bb-f14a-432b-a025-6bef2607e15b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:55:31.156837 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cd72bb-f14a-432b-a025-6bef2607e15b admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:55:31.238370 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:55:31.249761 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e81169e-c783-4557-9c93-db5724d9ce7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:55:31.250369 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e81169e-c783-4557-9c93-db5724d9ce7a admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:55:31.250640 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e81169e-c783-4557-9c93-db5724d9ce7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:55:31.250914 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e81169e-c783-4557-9c93-db5724d9ce7a admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:55:31.251697 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e81169e-c783-4557-9c93-db5724d9ce7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:55:31.252733 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: WARNING py.warnings [None req-7e81169e-c783-4557-9c93-db5724d9ce7a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 02 09:55:31.252733 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: warnings.warn(msg) Aug 02 09:55:31.252733 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: Aug 02 09:55:31.253787 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e81169e-c783-4557-9c93-db5724d9ce7a admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:55:31.266516 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 742/1499] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 09:55:31 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:55:31.270581 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9d89e5-aeb6-45c9-9d84-b699fcde4cb7 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:55:31.272094 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-9a9d89e5-aeb6-45c9-9d84-b699fcde4cb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:55:31.283985 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9d89e5-aeb6-45c9-9d84-b699fcde4cb7 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:55:31.308872 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:55:31.377844 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89488768-81cb-48f9-affa-84c6341a3ee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:55:31.378494 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89488768-81cb-48f9-affa-84c6341a3ee6 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:55:31.378778 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89488768-81cb-48f9-affa-84c6341a3ee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:55:31.379052 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89488768-81cb-48f9-affa-84c6341a3ee6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:55:31.379747 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89488768-81cb-48f9-affa-84c6341a3ee6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:55:31.380787 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: WARNING py.warnings [None req-89488768-81cb-48f9-affa-84c6341a3ee6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 02 09:55:31.380787 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: warnings.warn(msg) Aug 02 09:55:31.380787 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: Aug 02 09:55:31.381675 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89488768-81cb-48f9-affa-84c6341a3ee6 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:55:31.394166 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 758/1500] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 09:55:31 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:55:31.398506 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e81169e-c783-4557-9c93-db5724d9ce7a admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:55:31.400745 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-7e81169e-c783-4557-9c93-db5724d9ce7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:55:31.433021 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e81169e-c783-4557-9c93-db5724d9ce7a admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:55:31.526045 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a224f50-4512-4ff3-9c18-e8f553efa777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:55:31.526708 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a224f50-4512-4ff3-9c18-e8f553efa777 admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:55:31.526972 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a224f50-4512-4ff3-9c18-e8f553efa777 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:55:31.527243 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a224f50-4512-4ff3-9c18-e8f553efa777 admin admin] PATH_INFO: `/v3/projects/ad2cef6b05a74d6e8d2c38dd4791af11/users/15e6db5fc71d433ca818dcb7891140f8/roles/66c1074780954f8e87ed11a5c4e8c36a` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:55:31.548301 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a224f50-4512-4ff3-9c18-e8f553efa777 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad2cef6b05a74d6e8d2c38dd4791af11, user_id=15e6db5fc71d433ca818dcb7891140f8, role_id=66c1074780954f8e87ed11a5c4e8c36a)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:55:31.549335 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: WARNING py.warnings [None req-7a224f50-4512-4ff3-9c18-e8f553efa777 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 02 09:55:31.549335 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: warnings.warn(msg) Aug 02 09:55:31.549335 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: Aug 02 09:55:31.551240 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a224f50-4512-4ff3-9c18-e8f553efa777 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:55:31.583618 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 743/1501] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 09:55:31 2022] PUT /identity/v3/projects/ad2cef6b05a74d6e8d2c38dd4791af11/users/15e6db5fc71d433ca818dcb7891140f8/roles/66c1074780954f8e87ed11a5c4e8c36a => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:55:31.596561 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc8c646-777e-4914-ab60-17f3d4c7e335 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:55:31.596861 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc8c646-777e-4914-ab60-17f3d4c7e335 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:55:31.597146 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc8c646-777e-4914-ab60-17f3d4c7e335 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:55:31.652840 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-cdc8c646-777e-4914-ab60-17f3d4c7e335 None None] MFA Rules not processed for user `15e6db5fc71d433ca818dcb7891140f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:55:31.654028 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-cdc8c646-777e-4914-ab60-17f3d4c7e335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:55:31.729815 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 759/1502] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 09:55:31 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:55:31.742966 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a9b7c9-e983-41c3-a837-4a5057a1e0e2 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:55:31.743257 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a9b7c9-e983-41c3-a837-4a5057a1e0e2 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:55:31.743532 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a9b7c9-e983-41c3-a837-4a5057a1e0e2 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:55:31.744930 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 744/1503] 10.222.0.64 () {54 vars in 964 bytes} [Tue Aug 2 09:55:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:55:31.749737 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e697b0-649b-4274-ad9f-18037dfa7472 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:55:31.750034 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e697b0-649b-4274-ad9f-18037dfa7472 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:55:31.750321 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e697b0-649b-4274-ad9f-18037dfa7472 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:55:31.781445 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:55:31.829520 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-66e697b0-649b-4274-ad9f-18037dfa7472 None None] MFA Rules not processed for user `10d7de62b9ee4039bdc79627ac7fb499`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:55:31.830780 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-66e697b0-649b-4274-ad9f-18037dfa7472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:55:31.906649 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 760/1504] 10.222.0.64 () {58 vars in 1061 bytes} [Tue Aug 2 09:55:31 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:55:31.914355 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a9b7c9-e983-41c3-a837-4a5057a1e0e2 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:55:31.915919 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-b6a9b7c9-e983-41c3-a837-4a5057a1e0e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:55:31.966132 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:55:32.019213 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a9b7c9-e983-41c3-a837-4a5057a1e0e2 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:55:32.092097 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d3ae9a-be26-4277-bb99-0831cd59b1fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d7de62b9ee4039bdc79627ac7fb499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:55:32.092733 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d3ae9a-be26-4277-bb99-0831cd59b1fb service cinder] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:55:32.093023 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d3ae9a-be26-4277-bb99-0831cd59b1fb service cinder] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:55:32.093300 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d3ae9a-be26-4277-bb99-0831cd59b1fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:55:32.094930 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-67d3ae9a-be26-4277-bb99-0831cd59b1fb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:55:32.192208 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d3ae9a-be26-4277-bb99-0831cd59b1fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:55:32.193623 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d3ae9a-be26-4277-bb99-0831cd59b1fb service cinder] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:55:32.198163 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:55:32.214880 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 745/1505] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 09:55:31 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:55:32.268840 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97377324-2806-4675-9c61-465f271223b6 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:55:32.269133 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97377324-2806-4675-9c61-465f271223b6 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:55:32.269410 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97377324-2806-4675-9c61-465f271223b6 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:55:32.270675 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 761/1506] 10.222.0.64 () {52 vars in 830 bytes} [Tue Aug 2 09:55:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:55:32.274916 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d3ae9a-be26-4277-bb99-0831cd59b1fb service cinder] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:55:32.276154 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-67d3ae9a-be26-4277-bb99-0831cd59b1fb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:55:32.374462 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d3ae9a-be26-4277-bb99-0831cd59b1fb service cinder] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:55:32.470856 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859fc93d-4d5b-4441-86b8-cf44d6c86a73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:55:32.471412 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859fc93d-4d5b-4441-86b8-cf44d6c86a73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:55:32.471691 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859fc93d-4d5b-4441-86b8-cf44d6c86a73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:55:32.471963 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859fc93d-4d5b-4441-86b8-cf44d6c86a73 admin admin] PATH_INFO: `/v3/users/15e6db5fc71d433ca818dcb7891140f8` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:55:32.484567 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859fc93d-4d5b-4441-86b8-cf44d6c86a73 admin admin] RBAC: Authorizing `identity:delete_user(user_id=15e6db5fc71d433ca818dcb7891140f8)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:55:32.485625 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: WARNING py.warnings [None req-859fc93d-4d5b-4441-86b8-cf44d6c86a73 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 02 09:55:32.485625 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: warnings.warn(msg) Aug 02 09:55:32.485625 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: Aug 02 09:55:32.486824 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859fc93d-4d5b-4441-86b8-cf44d6c86a73 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:55:32.496261 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:55:32.552230 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-859fc93d-4d5b-4441-86b8-cf44d6c86a73 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15e6db5fc71d433ca818dcb7891140f8'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:55:32.565875 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-859fc93d-4d5b-4441-86b8-cf44d6c86a73 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15e6db5fc71d433ca818dcb7891140f8'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:55:32.566426 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-859fc93d-4d5b-4441-86b8-cf44d6c86a73 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15e6db5fc71d433ca818dcb7891140f8'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:55:32.575627 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-859fc93d-4d5b-4441-86b8-cf44d6c86a73 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15e6db5fc71d433ca818dcb7891140f8'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:55:32.598429 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-859fc93d-4d5b-4441-86b8-cf44d6c86a73 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15e6db5fc71d433ca818dcb7891140f8'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:55:32.604102 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 746/1507] 10.222.0.64 () {58 vars in 1224 bytes} [Tue Aug 2 09:55:32 2022] DELETE /identity/v3/users/15e6db5fc71d433ca818dcb7891140f8 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:55:32.608684 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685453df-5f2a-4dc9-9194-c43f1160389b None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:55:32.608973 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685453df-5f2a-4dc9-9194-c43f1160389b None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:55:32.609248 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685453df-5f2a-4dc9-9194-c43f1160389b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:55:32.664417 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-685453df-5f2a-4dc9-9194-c43f1160389b None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:55:32.666142 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-685453df-5f2a-4dc9-9194-c43f1160389b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:55:32.739441 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:55:32.753028 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 762/1508] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 09:55:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:55:32.762560 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859fc93d-4d5b-4441-86b8-cf44d6c86a73 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:55:32.763787 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-859fc93d-4d5b-4441-86b8-cf44d6c86a73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:55:32.861669 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859fc93d-4d5b-4441-86b8-cf44d6c86a73 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:55:32.916790 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:55:32.938619 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce9a3ae-92b2-445a-b48a-490674684f55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:55:32.939588 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce9a3ae-92b2-445a-b48a-490674684f55 service neutron] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:55:32.939937 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce9a3ae-92b2-445a-b48a-490674684f55 service neutron] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:55:32.940219 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce9a3ae-92b2-445a-b48a-490674684f55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:55:32.942241 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-0ce9a3ae-92b2-445a-b48a-490674684f55 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:55:33.049640 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce9a3ae-92b2-445a-b48a-490674684f55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:55:33.051044 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce9a3ae-92b2-445a-b48a-490674684f55 service neutron] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:55:33.069061 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 747/1509] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 09:55:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:55:33.433635 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685453df-5f2a-4dc9-9194-c43f1160389b None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:55:33.434906 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-685453df-5f2a-4dc9-9194-c43f1160389b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:55:33.446489 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685453df-5f2a-4dc9-9194-c43f1160389b None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:55:33.546593 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e33763-0289-4112-877c-9ce047313434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:55:33.547236 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e33763-0289-4112-877c-9ce047313434 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:55:33.547522 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e33763-0289-4112-877c-9ce047313434 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:55:33.547799 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e33763-0289-4112-877c-9ce047313434 admin admin] PATH_INFO: `/v3/projects/ad2cef6b05a74d6e8d2c38dd4791af11` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:55:33.558197 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e33763-0289-4112-877c-9ce047313434 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad2cef6b05a74d6e8d2c38dd4791af11)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:55:33.559233 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: WARNING py.warnings [None req-58e33763-0289-4112-877c-9ce047313434 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 02 09:55:33.559233 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: warnings.warn(msg) Aug 02 09:55:33.559233 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: Aug 02 09:55:33.560510 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e33763-0289-4112-877c-9ce047313434 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:55:33.621462 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-58e33763-0289-4112-877c-9ce047313434 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad2cef6b05a74d6e8d2c38dd4791af11', 'request_id': 'req-58e33763-0289-4112-877c-9ce047313434'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:55:33.627375 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-58e33763-0289-4112-877c-9ce047313434 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad2cef6b05a74d6e8d2c38dd4791af11', 'request_id': 'req-58e33763-0289-4112-877c-9ce047313434'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:55:33.642427 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-58e33763-0289-4112-877c-9ce047313434 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad2cef6b05a74d6e8d2c38dd4791af11', 'request_id': 'req-58e33763-0289-4112-877c-9ce047313434'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:55:33.669319 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-58e33763-0289-4112-877c-9ce047313434 admin admin] The token cache is being invalidate because project ad2cef6b05a74d6e8d2c38dd4791af11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80153) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:55:33.669622 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-58e33763-0289-4112-877c-9ce047313434 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:55:33.670054 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-58e33763-0289-4112-877c-9ce047313434 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:55:33.671522 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 763/1510] 10.222.0.64 () {58 vars in 1233 bytes} [Tue Aug 2 09:55:33 2022] DELETE /identity/v3/projects/ad2cef6b05a74d6e8d2c38dd4791af11 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:55:43.066743 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=62 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:55:43.667078 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:57:20.888747 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bef59d-4020-4366-aca0-3078593572a6 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:57:20.889368 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bef59d-4020-4366-aca0-3078593572a6 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:57:20.889368 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bef59d-4020-4366-aca0-3078593572a6 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:57:20.890631 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 748/1511] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 2 09:57:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:57:20.895671 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ded905e-ce24-45d4-800d-30287b6ccda0 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:57:20.896182 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ded905e-ce24-45d4-800d-30287b6ccda0 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:57:20.896395 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ded905e-ce24-45d4-800d-30287b6ccda0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:57:20.981371 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-4ded905e-ce24-45d4-800d-30287b6ccda0 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:57:20.982569 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-4ded905e-ce24-45d4-800d-30287b6ccda0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:57:21.083674 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 764/1512] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 2 09:57:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:57:21.235967 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bef59d-4020-4366-aca0-3078593572a6 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:57:21.238110 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-f5bef59d-4020-4366-aca0-3078593572a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:57:21.355444 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bef59d-4020-4366-aca0-3078593572a6 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:57:21.361346 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e532a29-b09f-4303-870b-e022d908eccd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99655e1f9ca24472a86cd26b0dbd1fc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:57:21.361966 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e532a29-b09f-4303-870b-e022d908eccd service nova] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:57:21.362255 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e532a29-b09f-4303-870b-e022d908eccd service nova] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:57:21.362529 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e532a29-b09f-4303-870b-e022d908eccd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:57:21.370441 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e532a29-b09f-4303-870b-e022d908eccd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:57:21.372651 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e532a29-b09f-4303-870b-e022d908eccd service nova] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:57:21.382809 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 749/1513] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 09:57:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:57:22.200597 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e303d1-9936-4170-9b8b-bf3fbd6ee745 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:57:22.201200 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e303d1-9936-4170-9b8b-bf3fbd6ee745 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:57:22.201200 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e303d1-9936-4170-9b8b-bf3fbd6ee745 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:57:22.202512 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 765/1514] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 2 09:57:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:57:22.207665 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06046dd-abc4-4ffa-bcdf-2257a7643b91 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:57:22.208119 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06046dd-abc4-4ffa-bcdf-2257a7643b91 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:57:22.208232 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06046dd-abc4-4ffa-bcdf-2257a7643b91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:57:22.227412 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-f06046dd-abc4-4ffa-bcdf-2257a7643b91 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:57:22.228649 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-f06046dd-abc4-4ffa-bcdf-2257a7643b91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:57:22.236962 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 750/1515] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 2 09:57:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:57:22.388750 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e303d1-9936-4170-9b8b-bf3fbd6ee745 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:57:22.404870 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e303d1-9936-4170-9b8b-bf3fbd6ee745 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:57:22.410478 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894d0f9f-a24a-47b9-89ac-a23eb60d81e3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99655e1f9ca24472a86cd26b0dbd1fc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:57:22.411082 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894d0f9f-a24a-47b9-89ac-a23eb60d81e3 service nova] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:57:22.411370 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894d0f9f-a24a-47b9-89ac-a23eb60d81e3 service nova] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:57:22.411651 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894d0f9f-a24a-47b9-89ac-a23eb60d81e3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:57:22.420301 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894d0f9f-a24a-47b9-89ac-a23eb60d81e3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:57:22.421898 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894d0f9f-a24a-47b9-89ac-a23eb60d81e3 service nova] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:57:22.431279 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 766/1516] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 09:57:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:57:22.867410 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d250a99-dfd8-4a95-920c-efe50a5cac9f None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:57:22.867744 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d250a99-dfd8-4a95-920c-efe50a5cac9f None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:57:22.868073 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d250a99-dfd8-4a95-920c-efe50a5cac9f None None] PATH_INFO: `/v3` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:57:22.869560 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 751/1517] 10.222.0.83 () {54 vars in 925 bytes} [Tue Aug 2 09:57:22 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:57:22.877702 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b82a11-6f73-4b9b-b76f-9639321a610a None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:57:22.877997 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b82a11-6f73-4b9b-b76f-9639321a610a None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:57:22.878274 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b82a11-6f73-4b9b-b76f-9639321a610a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:57:22.946784 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-f5b82a11-6f73-4b9b-b76f-9639321a610a None None] MFA Rules not processed for user `b770f073c86b4d88837a8b1f796a8b1d`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:57:22.948358 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-f5b82a11-6f73-4b9b-b76f-9639321a610a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:57:22.967070 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:57:23.021597 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 767/1518] 10.222.0.83 () {58 vars in 1015 bytes} [Tue Aug 2 09:57:22 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:57:32.222646 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=56 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:57:33.018712 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=23 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:57:33.648731 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4751d8-0f56-4a57-b5ee-331088844c4a None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:57:33.649220 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4751d8-0f56-4a57-b5ee-331088844c4a None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:57:33.649453 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4751d8-0f56-4a57-b5ee-331088844c4a None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:57:33.650834 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 752/1519] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 2 09:57:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 09:57:33.656993 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f431f78-7556-4710-93e3-24a46d3bb9fc None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:57:33.657292 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f431f78-7556-4710-93e3-24a46d3bb9fc None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:57:33.657577 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f431f78-7556-4710-93e3-24a46d3bb9fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:57:33.677533 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-6f431f78-7556-4710-93e3-24a46d3bb9fc None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:57:33.678750 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-6f431f78-7556-4710-93e3-24a46d3bb9fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:57:33.686729 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 768/1520] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 2 09:57:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:57:33.847864 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4751d8-0f56-4a57-b5ee-331088844c4a None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:57:33.864165 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4751d8-0f56-4a57-b5ee-331088844c4a None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:57:33.869735 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aeb65de-e331-4292-9fb9-c9b82b58f1d8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99655e1f9ca24472a86cd26b0dbd1fc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:57:33.870349 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeb65de-e331-4292-9fb9-c9b82b58f1d8 service nova] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:57:33.870633 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeb65de-e331-4292-9fb9-c9b82b58f1d8 service nova] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:57:33.870909 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeb65de-e331-4292-9fb9-c9b82b58f1d8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:57:33.879314 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aeb65de-e331-4292-9fb9-c9b82b58f1d8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:57:33.880997 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aeb65de-e331-4292-9fb9-c9b82b58f1d8 service nova] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:57:33.904306 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 753/1521] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 09:57:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:57:34.184853 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98818401-6147-475e-9948-ca44dd87880a None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:57:34.185967 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98818401-6147-475e-9948-ca44dd87880a None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:57:34.185967 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98818401-6147-475e-9948-ca44dd87880a None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:57:34.187298 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 769/1522] 10.222.0.64 () {54 vars in 961 bytes} [Tue Aug 2 09:57:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:57:34.192595 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866b0e2a-412b-47a6-b0bb-6481ba17387c None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:57:34.192892 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866b0e2a-412b-47a6-b0bb-6481ba17387c None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:57:34.193178 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866b0e2a-412b-47a6-b0bb-6481ba17387c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:57:34.212425 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-866b0e2a-412b-47a6-b0bb-6481ba17387c None None] MFA Rules not processed for user `b770f073c86b4d88837a8b1f796a8b1d`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:57:34.213628 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-866b0e2a-412b-47a6-b0bb-6481ba17387c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:57:34.222279 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 754/1523] 10.222.0.64 () {58 vars in 1058 bytes} [Tue Aug 2 09:57:34 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:57:34.229522 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98818401-6147-475e-9948-ca44dd87880a None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:57:34.243113 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98818401-6147-475e-9948-ca44dd87880a None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:57:34.248697 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9a1219-95d8-45c9-a2d9-8e1138d82bf6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b770f073c86b4d88837a8b1f796a8b1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:57:34.249300 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9a1219-95d8-45c9-a2d9-8e1138d82bf6 service placement] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:57:34.249594 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9a1219-95d8-45c9-a2d9-8e1138d82bf6 service placement] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:57:34.249873 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9a1219-95d8-45c9-a2d9-8e1138d82bf6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:57:34.258022 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9a1219-95d8-45c9-a2d9-8e1138d82bf6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:57:34.259593 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9a1219-95d8-45c9-a2d9-8e1138d82bf6 service placement] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:57:34.269307 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 770/1524] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 09:57:34 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:57:34.708301 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d29e3b-edbb-49a6-a78a-379d6232f910 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:57:34.708871 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d29e3b-edbb-49a6-a78a-379d6232f910 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:57:34.708871 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d29e3b-edbb-49a6-a78a-379d6232f910 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:57:34.710265 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 755/1525] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 2 09:57:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:57:34.714945 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccb1bd5-cd63-439b-8431-23edae0df3be None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:57:34.715232 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccb1bd5-cd63-439b-8431-23edae0df3be None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:57:34.715506 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccb1bd5-cd63-439b-8431-23edae0df3be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:57:34.735397 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-fccb1bd5-cd63-439b-8431-23edae0df3be None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:57:34.736612 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-fccb1bd5-cd63-439b-8431-23edae0df3be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:57:34.744262 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 771/1526] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 2 09:57:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:57:34.894177 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d29e3b-edbb-49a6-a78a-379d6232f910 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:57:34.909121 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d29e3b-edbb-49a6-a78a-379d6232f910 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:57:34.914564 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f050029-0567-4529-94be-7f1adeacb03c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99655e1f9ca24472a86cd26b0dbd1fc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:57:34.915168 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f050029-0567-4529-94be-7f1adeacb03c service nova] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:57:34.915662 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f050029-0567-4529-94be-7f1adeacb03c service nova] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:57:34.915940 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f050029-0567-4529-94be-7f1adeacb03c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:57:34.924687 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f050029-0567-4529-94be-7f1adeacb03c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:57:34.926424 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f050029-0567-4529-94be-7f1adeacb03c service nova] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:57:34.938626 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 756/1527] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 09:57:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:57:38.840683 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8217ac1-9ed9-4244-bd74-241334305596 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:57:38.841185 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8217ac1-9ed9-4244-bd74-241334305596 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:57:38.841284 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8217ac1-9ed9-4244-bd74-241334305596 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:57:38.842653 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 772/1528] 127.0.0.1 () {54 vars in 916 bytes} [Tue Aug 2 09:57:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:57:38.848104 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae6e178-a6fa-4aa5-95ad-dfe32bd90ebc None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:57:38.848614 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae6e178-a6fa-4aa5-95ad-dfe32bd90ebc None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:57:38.848723 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae6e178-a6fa-4aa5-95ad-dfe32bd90ebc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:57:38.868053 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-5ae6e178-a6fa-4aa5-95ad-dfe32bd90ebc None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:57:38.869529 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-5ae6e178-a6fa-4aa5-95ad-dfe32bd90ebc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:57:38.877941 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 757/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Aug 2 09:57:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:57:38.950153 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8217ac1-9ed9-4244-bd74-241334305596 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:57:38.951910 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-f8217ac1-9ed9-4244-bd74-241334305596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:57:39.029598 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8217ac1-9ed9-4244-bd74-241334305596 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:57:39.033268 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd6a7d0-5dbb-4222-9fdf-256d9ae85018 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:57:39.033871 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd6a7d0-5dbb-4222-9fdf-256d9ae85018 service neutron] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:57:39.034158 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd6a7d0-5dbb-4222-9fdf-256d9ae85018 service neutron] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:57:39.034432 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd6a7d0-5dbb-4222-9fdf-256d9ae85018 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:57:39.043176 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd6a7d0-5dbb-4222-9fdf-256d9ae85018 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:57:39.044707 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd6a7d0-5dbb-4222-9fdf-256d9ae85018 service neutron] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:57:39.054528 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 773/1530] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 09:57:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:57:41.052992 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564ef91c-8e1f-437d-8da5-373ce3489fe7 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:57:41.053589 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564ef91c-8e1f-437d-8da5-373ce3489fe7 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:57:41.053589 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564ef91c-8e1f-437d-8da5-373ce3489fe7 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:57:41.054913 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 758/1531] 127.0.0.1 () {54 vars in 916 bytes} [Tue Aug 2 09:57:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:57:41.060263 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f8c042-5779-4921-88dc-6923166f5964 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:57:41.060670 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f8c042-5779-4921-88dc-6923166f5964 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:57:41.060866 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f8c042-5779-4921-88dc-6923166f5964 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:57:41.080397 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-75f8c042-5779-4921-88dc-6923166f5964 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:57:41.081627 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-75f8c042-5779-4921-88dc-6923166f5964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:57:41.089723 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 774/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Aug 2 09:57:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:57:41.125149 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564ef91c-8e1f-437d-8da5-373ce3489fe7 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:57:41.135746 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564ef91c-8e1f-437d-8da5-373ce3489fe7 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:57:41.139750 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32694a1-354b-4f4d-a8c8-b3b571ffdaba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:57:41.140379 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32694a1-354b-4f4d-a8c8-b3b571ffdaba service neutron] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:57:41.140666 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32694a1-354b-4f4d-a8c8-b3b571ffdaba service neutron] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:57:41.140943 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32694a1-354b-4f4d-a8c8-b3b571ffdaba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:57:41.150391 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32694a1-354b-4f4d-a8c8-b3b571ffdaba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:57:41.151804 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32694a1-354b-4f4d-a8c8-b3b571ffdaba service neutron] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:57:41.162215 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 759/1533] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 09:57:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:57:44.851605 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a3a416-653e-449f-a51c-c0058caf27a5 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:57:44.852107 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a3a416-653e-449f-a51c-c0058caf27a5 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:57:44.852203 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a3a416-653e-449f-a51c-c0058caf27a5 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:57:44.853518 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 775/1534] 127.0.0.1 () {54 vars in 916 bytes} [Tue Aug 2 09:57:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:57:44.859112 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3570b8df-80c5-4cb1-a9f4-6a2d544f928e None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:57:44.859549 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3570b8df-80c5-4cb1-a9f4-6a2d544f928e None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:57:44.859657 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3570b8df-80c5-4cb1-a9f4-6a2d544f928e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:57:44.879263 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-3570b8df-80c5-4cb1-a9f4-6a2d544f928e None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:57:44.880483 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-3570b8df-80c5-4cb1-a9f4-6a2d544f928e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:57:44.888456 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 760/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Aug 2 09:57:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:57:45.197918 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a3a416-653e-449f-a51c-c0058caf27a5 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:57:45.199567 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-39a3a416-653e-449f-a51c-c0058caf27a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:57:45.277771 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a3a416-653e-449f-a51c-c0058caf27a5 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:57:45.281830 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748368d2-8f91-43e5-a26d-a06afde6ace8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d7de62b9ee4039bdc79627ac7fb499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:57:45.282440 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748368d2-8f91-43e5-a26d-a06afde6ace8 service cinder] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:57:45.282723 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748368d2-8f91-43e5-a26d-a06afde6ace8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:57:45.282996 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748368d2-8f91-43e5-a26d-a06afde6ace8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:57:45.291830 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748368d2-8f91-43e5-a26d-a06afde6ace8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:57:45.293295 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748368d2-8f91-43e5-a26d-a06afde6ace8 service cinder] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:57:45.303209 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 776/1536] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 09:57:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:57:47.307841 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d83c17e-99f3-4c1c-b225-116424d9dab0 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:57:47.308413 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d83c17e-99f3-4c1c-b225-116424d9dab0 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:57:47.308413 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d83c17e-99f3-4c1c-b225-116424d9dab0 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:57:47.309778 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 761/1537] 127.0.0.1 () {54 vars in 916 bytes} [Tue Aug 2 09:57:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:57:47.315633 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29b5eed-3483-45a0-a343-520e19992016 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:57:47.316066 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29b5eed-3483-45a0-a343-520e19992016 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:57:47.316179 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29b5eed-3483-45a0-a343-520e19992016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:57:47.337876 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-d29b5eed-3483-45a0-a343-520e19992016 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:57:47.339090 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-d29b5eed-3483-45a0-a343-520e19992016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:57:47.347172 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 777/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Aug 2 09:57:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:57:47.648902 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d83c17e-99f3-4c1c-b225-116424d9dab0 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:57:47.650574 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-7d83c17e-99f3-4c1c-b225-116424d9dab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:57:47.663912 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d83c17e-99f3-4c1c-b225-116424d9dab0 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:57:47.667376 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7132c9-10e3-463b-8899-db99795d4ce9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d7de62b9ee4039bdc79627ac7fb499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:57:47.667973 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7132c9-10e3-463b-8899-db99795d4ce9 service cinder] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:57:47.668283 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7132c9-10e3-463b-8899-db99795d4ce9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:57:47.668552 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7132c9-10e3-463b-8899-db99795d4ce9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:57:47.677905 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7132c9-10e3-463b-8899-db99795d4ce9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:57:47.679323 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7132c9-10e3-463b-8899-db99795d4ce9 service cinder] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:57:47.688484 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 762/1539] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 09:57:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:57:57.332983 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=88 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:57:57.684625 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=33 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:59:23.708740 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d536170d-62e2-49b8-9837-61686e2a6576 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:23.713289 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d536170d-62e2-49b8-9837-61686e2a6576 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:23.713289 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d536170d-62e2-49b8-9837-61686e2a6576 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:23.740193 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-d536170d-62e2-49b8-9837-61686e2a6576 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:59:23.742119 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-d536170d-62e2-49b8-9837-61686e2a6576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:23.761988 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 778/1540] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 09:59:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:59:23.774139 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109de21f-b74a-4fdc-8793-58423f266755 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:23.775185 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109de21f-b74a-4fdc-8793-58423f266755 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:23.775687 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109de21f-b74a-4fdc-8793-58423f266755 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:23.809591 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-109de21f-b74a-4fdc-8793-58423f266755 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:59:23.811852 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-109de21f-b74a-4fdc-8793-58423f266755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:23.830282 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 763/1541] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 09:59:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:59:23.835437 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d536170d-62e2-49b8-9837-61686e2a6576 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:23.854355 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d536170d-62e2-49b8-9837-61686e2a6576 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:23.870385 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d30947-1851-44e3-8d35-7aa3325b0252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:23.871402 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d30947-1851-44e3-8d35-7aa3325b0252 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:23.871929 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d30947-1851-44e3-8d35-7aa3325b0252 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:23.872447 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d30947-1851-44e3-8d35-7aa3325b0252 admin admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:23.873478 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d30947-1851-44e3-8d35-7aa3325b0252 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:59:23.874047 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d30947-1851-44e3-8d35-7aa3325b0252 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:23.875957 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d30947-1851-44e3-8d35-7aa3325b0252 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:23.900631 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 779/1542] 10.222.0.64 () {58 vars in 1153 bytes} [Tue Aug 2 09:59:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:59:23.905548 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109de21f-b74a-4fdc-8793-58423f266755 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:23.918385 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109de21f-b74a-4fdc-8793-58423f266755 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:23.926304 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6ca3c4-44eb-47df-9c03-1a9aad410f6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:23.927231 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6ca3c4-44eb-47df-9c03-1a9aad410f6c admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:23.927730 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6ca3c4-44eb-47df-9c03-1a9aad410f6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:23.928216 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6ca3c4-44eb-47df-9c03-1a9aad410f6c admin admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:23.929142 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6ca3c4-44eb-47df-9c03-1a9aad410f6c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:23.931262 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6ca3c4-44eb-47df-9c03-1a9aad410f6c admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:24.011819 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 764/1543] 10.222.0.64 () {60 vars in 1153 bytes} [Tue Aug 2 09:59:23 2022] POST /identity/v3/projects => generated 393 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:59:24.017210 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d30947-1851-44e3-8d35-7aa3325b0252 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:24.035590 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d30947-1851-44e3-8d35-7aa3325b0252 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:24.119123 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e161ce3-5a35-4f39-818f-7c6d0adfc912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:24.120327 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e161ce3-5a35-4f39-818f-7c6d0adfc912 admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:24.120848 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e161ce3-5a35-4f39-818f-7c6d0adfc912 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:24.121345 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e161ce3-5a35-4f39-818f-7c6d0adfc912 admin admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:24.122244 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e161ce3-5a35-4f39-818f-7c6d0adfc912 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:24.125115 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e161ce3-5a35-4f39-818f-7c6d0adfc912 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:24.171589 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 780/1544] 10.222.0.64 () {60 vars in 1144 bytes} [Tue Aug 2 09:59:24 2022] POST /identity/v3/users => generated 297 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:59:24.176139 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6ca3c4-44eb-47df-9c03-1a9aad410f6c admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:24.188677 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6ca3c4-44eb-47df-9c03-1a9aad410f6c admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:24.196757 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc35fb60-a92d-4fd1-9a58-67c65cb9185f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:24.197522 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc35fb60-a92d-4fd1-9a58-67c65cb9185f admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:24.197817 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc35fb60-a92d-4fd1-9a58-67c65cb9185f admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:24.198100 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc35fb60-a92d-4fd1-9a58-67c65cb9185f admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:24.199577 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc35fb60-a92d-4fd1-9a58-67c65cb9185f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:24.201322 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc35fb60-a92d-4fd1-9a58-67c65cb9185f admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:24.210721 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6241b72c-6727-458b-9489-89c45f585667 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:24.211313 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6241b72c-6727-458b-9489-89c45f585667 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:24.212349 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6241b72c-6727-458b-9489-89c45f585667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:24.223098 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 765/1545] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 09:59:24 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:59:24.224238 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc35fb60-a92d-4fd1-9a58-67c65cb9185f admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:24.240271 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc35fb60-a92d-4fd1-9a58-67c65cb9185f admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:24.243941 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-6241b72c-6727-458b-9489-89c45f585667 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:59:24.244630 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f23eca-a5c7-43ae-a8cb-edcf50d1d92c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:24.245269 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f23eca-a5c7-43ae-a8cb-edcf50d1d92c admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:24.245561 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f23eca-a5c7-43ae-a8cb-edcf50d1d92c admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:24.245839 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f23eca-a5c7-43ae-a8cb-edcf50d1d92c admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:24.246405 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-6241b72c-6727-458b-9489-89c45f585667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:24.246533 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f23eca-a5c7-43ae-a8cb-edcf50d1d92c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:24.252127 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f23eca-a5c7-43ae-a8cb-edcf50d1d92c admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:24.270162 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 781/1546] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 09:59:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:59:24.276051 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 766/1547] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 09:59:24 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 09:59:24.280311 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6241b72c-6727-458b-9489-89c45f585667 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:24.281493 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6127f3aa-19a2-45cf-8a96-d702eb89074c None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:24.281790 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6127f3aa-19a2-45cf-8a96-d702eb89074c None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:24.282072 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6127f3aa-19a2-45cf-8a96-d702eb89074c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:24.296320 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6241b72c-6727-458b-9489-89c45f585667 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:24.309896 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d0f404-33ea-460d-8368-a9d8d0fae4c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:24.311546 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d0f404-33ea-460d-8368-a9d8d0fae4c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:24.312052 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d0f404-33ea-460d-8368-a9d8d0fae4c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:24.312582 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d0f404-33ea-460d-8368-a9d8d0fae4c2 admin admin] PATH_INFO: `/v3/projects/ec62d727071640d8b76345246bb7be13/users/5bbba5ae48cc4ec39467df736ad1bbad/roles/66c1074780954f8e87ed11a5c4e8c36a` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:24.317486 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-6127f3aa-19a2-45cf-8a96-d702eb89074c None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:59:24.319409 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-6127f3aa-19a2-45cf-8a96-d702eb89074c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:24.331131 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d0f404-33ea-460d-8368-a9d8d0fae4c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec62d727071640d8b76345246bb7be13, user_id=5bbba5ae48cc4ec39467df736ad1bbad, role_id=66c1074780954f8e87ed11a5c4e8c36a)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:24.332514 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: WARNING py.warnings [None req-77d0f404-33ea-460d-8368-a9d8d0fae4c2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 02 09:59:24.332514 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: warnings.warn(msg) Aug 02 09:59:24.332514 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: Aug 02 09:59:24.335146 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d0f404-33ea-460d-8368-a9d8d0fae4c2 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:24.335275 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 767/1548] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 09:59:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:59:24.337165 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7459d489-fec3-4a13-a76a-d8951298d479 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:24.337460 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7459d489-fec3-4a13-a76a-d8951298d479 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:24.337747 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7459d489-fec3-4a13-a76a-d8951298d479 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:24.361359 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-7459d489-fec3-4a13-a76a-d8951298d479 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:59:24.362993 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-7459d489-fec3-4a13-a76a-d8951298d479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:24.378742 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 782/1549] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 09:59:24 2022] PUT /identity/v3/projects/ec62d727071640d8b76345246bb7be13/users/5bbba5ae48cc4ec39467df736ad1bbad/roles/66c1074780954f8e87ed11a5c4e8c36a => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:59:24.381246 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0705dfc0-4c7c-4d20-b835-d3b0990c9b27 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:24.381886 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0705dfc0-4c7c-4d20-b835-d3b0990c9b27 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:24.382378 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0705dfc0-4c7c-4d20-b835-d3b0990c9b27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:24.396071 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 768/1550] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 09:59:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:59:24.399350 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7459d489-fec3-4a13-a76a-d8951298d479 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:24.410134 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7459d489-fec3-4a13-a76a-d8951298d479 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:24.424292 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-0705dfc0-4c7c-4d20-b835-d3b0990c9b27 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:59:24.425617 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-0705dfc0-4c7c-4d20-b835-d3b0990c9b27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:24.513974 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47529d8d-c6a1-4b61-a48e-4b21faba2bc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:24.514684 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47529d8d-c6a1-4b61-a48e-4b21faba2bc2 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:24.514974 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47529d8d-c6a1-4b61-a48e-4b21faba2bc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:24.515251 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47529d8d-c6a1-4b61-a48e-4b21faba2bc2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:24.515959 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47529d8d-c6a1-4b61-a48e-4b21faba2bc2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:59:24.516302 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47529d8d-c6a1-4b61-a48e-4b21faba2bc2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:24.517330 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: WARNING py.warnings [None req-47529d8d-c6a1-4b61-a48e-4b21faba2bc2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 02 09:59:24.517330 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: warnings.warn(msg) Aug 02 09:59:24.517330 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: Aug 02 09:59:24.518332 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47529d8d-c6a1-4b61-a48e-4b21faba2bc2 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:24.527824 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 783/1551] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 09:59:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:59:24.536270 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 769/1552] 10.222.0.64 () {58 vars in 1153 bytes} [Tue Aug 2 09:59:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:59:24.536270 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a0a315-a5b8-4944-81a8-b7ae00ef4c51 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:24.536270 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a0a315-a5b8-4944-81a8-b7ae00ef4c51 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:24.536616 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d192c7-53b1-46fd-b462-35938c22da19 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:24.536713 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a0a315-a5b8-4944-81a8-b7ae00ef4c51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:24.536796 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d192c7-53b1-46fd-b462-35938c22da19 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:24.536796 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d192c7-53b1-46fd-b462-35938c22da19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:24.554073 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-86d192c7-53b1-46fd-b462-35938c22da19 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:59:24.554251 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-85a0a315-a5b8-4944-81a8-b7ae00ef4c51 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:59:24.555265 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-86d192c7-53b1-46fd-b462-35938c22da19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:24.555380 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-85a0a315-a5b8-4944-81a8-b7ae00ef4c51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:24.563504 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 784/1553] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 09:59:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:59:24.564021 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 770/1554] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 09:59:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:59:24.564849 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bb839f-8db6-42b6-8437-2f5b77f32a09 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:24.565125 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bb839f-8db6-42b6-8437-2f5b77f32a09 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:24.565397 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bb839f-8db6-42b6-8437-2f5b77f32a09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:24.567911 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c764cb43-94dd-4e5b-acfc-0b208a9e9a1f None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:24.568176 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c764cb43-94dd-4e5b-acfc-0b208a9e9a1f None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:24.568586 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c764cb43-94dd-4e5b-acfc-0b208a9e9a1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:24.584919 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-c2bb839f-8db6-42b6-8437-2f5b77f32a09 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:59:24.586126 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-c2bb839f-8db6-42b6-8437-2f5b77f32a09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:24.590036 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-c764cb43-94dd-4e5b-acfc-0b208a9e9a1f None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:59:24.591249 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-c764cb43-94dd-4e5b-acfc-0b208a9e9a1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:24.593944 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 785/1555] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 09:59:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:59:24.595108 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf78c9c-bded-413c-89a3-c3d52bbb39d4 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:24.595393 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf78c9c-bded-413c-89a3-c3d52bbb39d4 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:24.595665 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf78c9c-bded-413c-89a3-c3d52bbb39d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:24.599543 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 771/1556] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 09:59:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:59:24.600617 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a150decc-2c18-4f18-b94f-11ba69bdbdcc None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:24.600922 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a150decc-2c18-4f18-b94f-11ba69bdbdcc None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:24.601182 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a150decc-2c18-4f18-b94f-11ba69bdbdcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:24.620277 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-a150decc-2c18-4f18-b94f-11ba69bdbdcc None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:59:24.621451 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-a150decc-2c18-4f18-b94f-11ba69bdbdcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:24.622722 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-ebf78c9c-bded-413c-89a3-c3d52bbb39d4 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:59:24.624459 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-ebf78c9c-bded-413c-89a3-c3d52bbb39d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:24.629955 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 772/1557] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 09:59:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:59:24.631114 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea758d9-2f40-41f4-97c7-d060bbc40709 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:24.631404 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea758d9-2f40-41f4-97c7-d060bbc40709 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:24.631679 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea758d9-2f40-41f4-97c7-d060bbc40709 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:24.637892 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 786/1558] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 09:59:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:59:24.641150 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf78c9c-bded-413c-89a3-c3d52bbb39d4 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:24.650340 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf78c9c-bded-413c-89a3-c3d52bbb39d4 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:24.651937 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-fea758d9-2f40-41f4-97c7-d060bbc40709 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:59:24.653186 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-fea758d9-2f40-41f4-97c7-d060bbc40709 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:24.655068 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1007910b-a333-4b50-a7d8-e8915b750a9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:24.655679 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1007910b-a333-4b50-a7d8-e8915b750a9c admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:24.655962 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1007910b-a333-4b50-a7d8-e8915b750a9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:24.656233 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1007910b-a333-4b50-a7d8-e8915b750a9c admin admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:24.656914 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1007910b-a333-4b50-a7d8-e8915b750a9c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:24.657935 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: WARNING py.warnings [None req-1007910b-a333-4b50-a7d8-e8915b750a9c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 02 09:59:24.657935 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: warnings.warn(msg) Aug 02 09:59:24.657935 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: Aug 02 09:59:24.659112 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1007910b-a333-4b50-a7d8-e8915b750a9c admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:24.661124 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 773/1559] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 09:59:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:59:24.662301 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba10e95-9346-4caa-b496-b5aa17a316f0 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:24.662593 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba10e95-9346-4caa-b496-b5aa17a316f0 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:24.662884 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba10e95-9346-4caa-b496-b5aa17a316f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:24.681863 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:59:24.683003 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-aba10e95-9346-4caa-b496-b5aa17a316f0 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:59:24.684360 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-aba10e95-9346-4caa-b496-b5aa17a316f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:24.690433 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 787/1560] 10.222.0.64 () {60 vars in 1153 bytes} [Tue Aug 2 09:59:24 2022] POST /identity/v3/projects => generated 377 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:59:24.691330 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1007910b-a333-4b50-a7d8-e8915b750a9c admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:24.695150 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 774/1561] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 09:59:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:59:24.698796 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba10e95-9346-4caa-b496-b5aa17a316f0 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:24.705811 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba10e95-9346-4caa-b496-b5aa17a316f0 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:24.750972 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:59:24.751785 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1007910b-a333-4b50-a7d8-e8915b750a9c admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:24.757065 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0b7139-9b47-4b39-830b-bfccbef7d434 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:24.757726 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0b7139-9b47-4b39-830b-bfccbef7d434 service neutron] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:24.758013 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0b7139-9b47-4b39-830b-bfccbef7d434 service neutron] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:24.758289 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0b7139-9b47-4b39-830b-bfccbef7d434 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:24.765242 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc61a11-c501-4295-8e31-eca443f2a496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:24.766188 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc61a11-c501-4295-8e31-eca443f2a496 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:24.766478 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc61a11-c501-4295-8e31-eca443f2a496 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:24.766758 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc61a11-c501-4295-8e31-eca443f2a496 admin admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:24.767490 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc61a11-c501-4295-8e31-eca443f2a496 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:59:24.767801 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc61a11-c501-4295-8e31-eca443f2a496 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:24.769559 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc61a11-c501-4295-8e31-eca443f2a496 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:24.782049 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 775/1562] 10.222.0.64 () {58 vars in 1153 bytes} [Tue Aug 2 09:59:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:59:24.783373 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62865767-109e-41c4-9402-bd06748f8873 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:24.783668 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62865767-109e-41c4-9402-bd06748f8873 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:24.784496 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62865767-109e-41c4-9402-bd06748f8873 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:24.812410 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-62865767-109e-41c4-9402-bd06748f8873 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:59:24.813613 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-62865767-109e-41c4-9402-bd06748f8873 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:24.821492 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 776/1563] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 09:59:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:59:24.822672 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b244a025-4415-48f4-b9f1-47ec1f98fff5 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:24.822957 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b244a025-4415-48f4-b9f1-47ec1f98fff5 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:24.823231 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b244a025-4415-48f4-b9f1-47ec1f98fff5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:24.824228 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0b7139-9b47-4b39-830b-bfccbef7d434 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:24.825851 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0b7139-9b47-4b39-830b-bfccbef7d434 service neutron] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:24.836398 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 788/1564] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 09:59:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:59:24.837628 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a406002-d551-4985-848e-fb7713582909 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:24.837918 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a406002-d551-4985-848e-fb7713582909 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:24.838195 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a406002-d551-4985-848e-fb7713582909 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:24.843024 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-b244a025-4415-48f4-b9f1-47ec1f98fff5 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:59:24.844221 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-b244a025-4415-48f4-b9f1-47ec1f98fff5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:24.852179 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 777/1565] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 09:59:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:59:24.853003 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b244a025-4415-48f4-b9f1-47ec1f98fff5 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:24.863216 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b244a025-4415-48f4-b9f1-47ec1f98fff5 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:24.863570 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-1a406002-d551-4985-848e-fb7713582909 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:59:24.865732 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-1a406002-d551-4985-848e-fb7713582909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:24.868010 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c4de08-5c72-41ac-8136-0c090ad67902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:24.868643 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c4de08-5c72-41ac-8136-0c090ad67902 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:24.868970 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c4de08-5c72-41ac-8136-0c090ad67902 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:24.869322 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c4de08-5c72-41ac-8136-0c090ad67902 admin admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:24.870045 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c4de08-5c72-41ac-8136-0c090ad67902 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:59:24.870351 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c4de08-5c72-41ac-8136-0c090ad67902 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:24.872117 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c4de08-5c72-41ac-8136-0c090ad67902 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:24.875215 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 789/1566] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 09:59:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:59:24.876129 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a406002-d551-4985-848e-fb7713582909 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:24.896329 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 778/1567] 10.222.0.64 () {58 vars in 1153 bytes} [Tue Aug 2 09:59:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:59:24.896524 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a406002-d551-4985-848e-fb7713582909 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:24.896938 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c4de08-5c72-41ac-8136-0c090ad67902 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:24.901258 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ebb1c1-49bc-4e36-a466-72b55f1eca30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:24.901947 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ebb1c1-49bc-4e36-a466-72b55f1eca30 admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:24.902066 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c4de08-5c72-41ac-8136-0c090ad67902 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:24.902242 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ebb1c1-49bc-4e36-a466-72b55f1eca30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:24.902542 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ebb1c1-49bc-4e36-a466-72b55f1eca30 admin admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:24.903271 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ebb1c1-49bc-4e36-a466-72b55f1eca30 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:24.906114 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ebb1c1-49bc-4e36-a466-72b55f1eca30 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:24.908213 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822da98b-a02c-4705-afd0-e1ad6c35a812 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:24.909020 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822da98b-a02c-4705-afd0-e1ad6c35a812 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:24.909364 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822da98b-a02c-4705-afd0-e1ad6c35a812 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:24.909641 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822da98b-a02c-4705-afd0-e1ad6c35a812 admin admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:24.910365 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822da98b-a02c-4705-afd0-e1ad6c35a812 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:59:24.910682 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822da98b-a02c-4705-afd0-e1ad6c35a812 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:24.912472 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822da98b-a02c-4705-afd0-e1ad6c35a812 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:24.929314 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 779/1568] 10.222.0.64 () {58 vars in 1153 bytes} [Tue Aug 2 09:59:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:59:24.930310 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822da98b-a02c-4705-afd0-e1ad6c35a812 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:24.935741 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 790/1569] 10.222.0.64 () {60 vars in 1144 bytes} [Tue Aug 2 09:59:24 2022] POST /identity/v3/users => generated 289 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:59:24.935741 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ebb1c1-49bc-4e36-a466-72b55f1eca30 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:24.941384 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822da98b-a02c-4705-afd0-e1ad6c35a812 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:24.945572 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ebb1c1-49bc-4e36-a466-72b55f1eca30 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:24.946650 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e092b038-2912-4d42-9c97-fb883f9d0f28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:24.947192 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e092b038-2912-4d42-9c97-fb883f9d0f28 admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:24.947537 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e092b038-2912-4d42-9c97-fb883f9d0f28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:24.947976 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e092b038-2912-4d42-9c97-fb883f9d0f28 admin admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:24.948758 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e092b038-2912-4d42-9c97-fb883f9d0f28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:24.950832 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e092b038-2912-4d42-9c97-fb883f9d0f28 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:24.951279 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47d0297-49c1-48c4-9569-27771f07b02f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:24.952114 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47d0297-49c1-48c4-9569-27771f07b02f admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:24.952505 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47d0297-49c1-48c4-9569-27771f07b02f admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:24.952828 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47d0297-49c1-48c4-9569-27771f07b02f admin admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:24.954280 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47d0297-49c1-48c4-9569-27771f07b02f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:59:24.954463 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47d0297-49c1-48c4-9569-27771f07b02f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:24.956164 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47d0297-49c1-48c4-9569-27771f07b02f admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:24.972022 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 791/1570] 10.222.0.64 () {58 vars in 1153 bytes} [Tue Aug 2 09:59:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:59:24.973106 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47d0297-49c1-48c4-9569-27771f07b02f admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:24.984827 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47d0297-49c1-48c4-9569-27771f07b02f admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:24.988873 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 780/1571] 10.222.0.64 () {60 vars in 1153 bytes} [Tue Aug 2 09:59:24 2022] POST /identity/v3/projects => generated 371 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:59:24.990078 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33766cc4-f445-45f6-be1d-b1d5ab476958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:24.990793 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33766cc4-f445-45f6-be1d-b1d5ab476958 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:24.991122 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33766cc4-f445-45f6-be1d-b1d5ab476958 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:24.991545 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33766cc4-f445-45f6-be1d-b1d5ab476958 admin admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:24.992453 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33766cc4-f445-45f6-be1d-b1d5ab476958 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:59:24.992748 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33766cc4-f445-45f6-be1d-b1d5ab476958 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:24.992940 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e092b038-2912-4d42-9c97-fb883f9d0f28 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:24.994353 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33766cc4-f445-45f6-be1d-b1d5ab476958 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:25.009231 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 792/1572] 10.222.0.64 () {58 vars in 1153 bytes} [Tue Aug 2 09:59:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:59:25.017361 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33766cc4-f445-45f6-be1d-b1d5ab476958 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:25.029687 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33766cc4-f445-45f6-be1d-b1d5ab476958 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:25.070787 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e092b038-2912-4d42-9c97-fb883f9d0f28 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:25.077442 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e13d304-3029-4aee-9687-c43246095dd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:25.078079 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e13d304-3029-4aee-9687-c43246095dd0 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:25.078367 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e13d304-3029-4aee-9687-c43246095dd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:25.078644 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e13d304-3029-4aee-9687-c43246095dd0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:25.079640 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e13d304-3029-4aee-9687-c43246095dd0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:59:25.079948 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e13d304-3029-4aee-9687-c43246095dd0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:25.082095 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e13d304-3029-4aee-9687-c43246095dd0 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:25.096358 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 781/1573] 10.222.0.64 () {58 vars in 1153 bytes} [Tue Aug 2 09:59:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:59:25.096579 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e13d304-3029-4aee-9687-c43246095dd0 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:25.111567 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:59:25.115440 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e13d304-3029-4aee-9687-c43246095dd0 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:25.121372 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9438f5dd-c478-4a21-856f-0910ac740518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:25.122241 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9438f5dd-c478-4a21-856f-0910ac740518 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:25.122529 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9438f5dd-c478-4a21-856f-0910ac740518 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:25.122808 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9438f5dd-c478-4a21-856f-0910ac740518 admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:25.123538 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9438f5dd-c478-4a21-856f-0910ac740518 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:25.124663 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e4a473-6075-4362-838a-69133e0cd6c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:25.125197 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9438f5dd-c478-4a21-856f-0910ac740518 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:25.125627 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e4a473-6075-4362-838a-69133e0cd6c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:25.125979 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e4a473-6075-4362-838a-69133e0cd6c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:25.126437 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e4a473-6075-4362-838a-69133e0cd6c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:25.127496 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e4a473-6075-4362-838a-69133e0cd6c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:25.130364 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e4a473-6075-4362-838a-69133e0cd6c0 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:25.137422 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 782/1574] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 09:59:25 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:59:25.138503 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9438f5dd-c478-4a21-856f-0910ac740518 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:25.149166 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9438f5dd-c478-4a21-856f-0910ac740518 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:25.155968 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5d53ea-f402-4794-b47a-46095edf78a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:25.156847 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5d53ea-f402-4794-b47a-46095edf78a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:25.157213 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5d53ea-f402-4794-b47a-46095edf78a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:25.157496 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5d53ea-f402-4794-b47a-46095edf78a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:25.158253 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5d53ea-f402-4794-b47a-46095edf78a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:25.160433 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5d53ea-f402-4794-b47a-46095edf78a6 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:25.173497 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 793/1575] 10.222.0.64 () {60 vars in 1153 bytes} [Tue Aug 2 09:59:25 2022] POST /identity/v3/projects => generated 389 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:59:25.174831 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e4a473-6075-4362-838a-69133e0cd6c0 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:25.194829 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 783/1576] 10.222.0.64 () {60 vars in 1153 bytes} [Tue Aug 2 09:59:25 2022] POST /identity/v3/projects => generated 383 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:59:25.196035 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5d53ea-f402-4794-b47a-46095edf78a6 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:25.263213 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:59:25.266226 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e4a473-6075-4362-838a-69133e0cd6c0 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:25.296929 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5d53ea-f402-4794-b47a-46095edf78a6 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:25.303312 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ee0c5c-1e88-46c1-b258-669bd7e24985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:25.304006 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ee0c5c-1e88-46c1-b258-669bd7e24985 admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:25.304498 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ee0c5c-1e88-46c1-b258-669bd7e24985 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:25.304813 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ee0c5c-1e88-46c1-b258-669bd7e24985 admin admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:25.306309 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ee0c5c-1e88-46c1-b258-669bd7e24985 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:25.307800 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: WARNING py.warnings [None req-c9ee0c5c-1e88-46c1-b258-669bd7e24985 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 02 09:59:25.307800 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: warnings.warn(msg) Aug 02 09:59:25.307800 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: Aug 02 09:59:25.309118 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ee0c5c-1e88-46c1-b258-669bd7e24985 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:25.344284 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 784/1577] 10.222.0.64 () {60 vars in 1144 bytes} [Tue Aug 2 09:59:25 2022] POST /identity/v3/users => generated 286 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:59:25.344284 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ee0c5c-1e88-46c1-b258-669bd7e24985 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:25.347894 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62db4d8-0d23-4964-ab65-5a542dc17d27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:25.348608 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62db4d8-0d23-4964-ab65-5a542dc17d27 admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:25.348956 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62db4d8-0d23-4964-ab65-5a542dc17d27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:25.349235 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62db4d8-0d23-4964-ab65-5a542dc17d27 admin admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:25.349945 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62db4d8-0d23-4964-ab65-5a542dc17d27 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:25.352001 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62db4d8-0d23-4964-ab65-5a542dc17d27 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:25.353347 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ee0c5c-1e88-46c1-b258-669bd7e24985 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:25.359077 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d6706f-d083-4a3b-8dba-3b18f2175b72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:25.359700 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d6706f-d083-4a3b-8dba-3b18f2175b72 admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:25.359985 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d6706f-d083-4a3b-8dba-3b18f2175b72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:25.360284 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d6706f-d083-4a3b-8dba-3b18f2175b72 admin admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:25.360965 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d6706f-d083-4a3b-8dba-3b18f2175b72 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:25.363799 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d6706f-d083-4a3b-8dba-3b18f2175b72 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:25.384371 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 794/1578] 10.222.0.64 () {60 vars in 1153 bytes} [Tue Aug 2 09:59:25 2022] POST /identity/v3/projects => generated 375 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:59:25.384371 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62db4d8-0d23-4964-ab65-5a542dc17d27 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:25.399154 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 785/1579] 10.222.0.64 () {60 vars in 1153 bytes} [Tue Aug 2 09:59:25 2022] POST /identity/v3/projects => generated 377 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:59:25.400226 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d6706f-d083-4a3b-8dba-3b18f2175b72 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:25.462580 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62db4d8-0d23-4964-ab65-5a542dc17d27 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:25.475647 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d6706f-d083-4a3b-8dba-3b18f2175b72 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:25.481204 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a806bf6-eeac-4a06-b609-0b3aa99d30b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:25.481855 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a806bf6-eeac-4a06-b609-0b3aa99d30b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:25.482106 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a806bf6-eeac-4a06-b609-0b3aa99d30b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:25.482371 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a806bf6-eeac-4a06-b609-0b3aa99d30b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:25.482882 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:59:25.483225 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a806bf6-eeac-4a06-b609-0b3aa99d30b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:25.484970 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a806bf6-eeac-4a06-b609-0b3aa99d30b2 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:25.496297 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 786/1580] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 09:59:25 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:59:25.497309 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a806bf6-eeac-4a06-b609-0b3aa99d30b2 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:25.508550 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a806bf6-eeac-4a06-b609-0b3aa99d30b2 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:25.517007 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ee661d-abef-420d-8544-37a9b9ba4ecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:25.518020 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ee661d-abef-420d-8544-37a9b9ba4ecd admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:25.518304 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ee661d-abef-420d-8544-37a9b9ba4ecd admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:25.518576 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ee661d-abef-420d-8544-37a9b9ba4ecd admin admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:25.519303 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ee661d-abef-420d-8544-37a9b9ba4ecd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:25.521262 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ee661d-abef-420d-8544-37a9b9ba4ecd admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:25.540429 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2f8d50-d801-4040-9ac0-2bf8468f231b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:25.541175 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2f8d50-d801-4040-9ac0-2bf8468f231b admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:25.541619 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2f8d50-d801-4040-9ac0-2bf8468f231b admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:25.541976 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2f8d50-d801-4040-9ac0-2bf8468f231b admin admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:25.542773 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2f8d50-d801-4040-9ac0-2bf8468f231b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:25.544862 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2f8d50-d801-4040-9ac0-2bf8468f231b admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:25.552823 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 787/1581] 10.222.0.64 () {60 vars in 1144 bytes} [Tue Aug 2 09:59:25 2022] POST /identity/v3/users => generated 295 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:59:25.553583 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ee661d-abef-420d-8544-37a9b9ba4ecd admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:25.565947 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ee661d-abef-420d-8544-37a9b9ba4ecd admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:25.571564 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb027c62-56c9-43c2-b158-4942278f744e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:25.572264 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb027c62-56c9-43c2-b158-4942278f744e admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:25.572568 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb027c62-56c9-43c2-b158-4942278f744e admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:25.572885 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb027c62-56c9-43c2-b158-4942278f744e admin admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:25.573566 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb027c62-56c9-43c2-b158-4942278f744e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:25.575540 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb027c62-56c9-43c2-b158-4942278f744e admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:25.582235 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 795/1582] 10.222.0.64 () {60 vars in 1153 bytes} [Tue Aug 2 09:59:25 2022] POST /identity/v3/projects => generated 383 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:59:25.583272 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2f8d50-d801-4040-9ac0-2bf8468f231b admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:25.620302 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 788/1583] 10.222.0.64 () {60 vars in 1144 bytes} [Tue Aug 2 09:59:25 2022] POST /identity/v3/users => generated 292 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:59:25.621638 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb027c62-56c9-43c2-b158-4942278f744e admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:25.638374 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb027c62-56c9-43c2-b158-4942278f744e admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:25.660171 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2f8d50-d801-4040-9ac0-2bf8468f231b admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:25.665696 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b531b3-d717-413d-bd1d-99ed3434ded4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:25.666532 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b531b3-d717-413d-bd1d-99ed3434ded4 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:25.667077 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b531b3-d717-413d-bd1d-99ed3434ded4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:25.667519 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b531b3-d717-413d-bd1d-99ed3434ded4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:25.668461 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b531b3-d717-413d-bd1d-99ed3434ded4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:25.670317 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b531b3-d717-413d-bd1d-99ed3434ded4 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:25.681510 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 796/1584] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 09:59:25 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:59:25.682615 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b531b3-d717-413d-bd1d-99ed3434ded4 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:25.694546 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b531b3-d717-413d-bd1d-99ed3434ded4 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:25.699955 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1403274-7b0f-441e-a6d5-e6c39a26b42a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:25.701016 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1403274-7b0f-441e-a6d5-e6c39a26b42a admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:25.701386 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1403274-7b0f-441e-a6d5-e6c39a26b42a admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:25.702342 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1403274-7b0f-441e-a6d5-e6c39a26b42a admin admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:25.703277 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1403274-7b0f-441e-a6d5-e6c39a26b42a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:25.706034 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1403274-7b0f-441e-a6d5-e6c39a26b42a admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:25.735655 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 797/1585] 10.222.0.64 () {60 vars in 1144 bytes} [Tue Aug 2 09:59:25 2022] POST /identity/v3/users => generated 289 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:59:25.736898 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1403274-7b0f-441e-a6d5-e6c39a26b42a admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:25.740041 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52ea1eb-ed15-4161-9671-5c3cb67dc02e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:25.740756 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52ea1eb-ed15-4161-9671-5c3cb67dc02e admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:25.741105 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52ea1eb-ed15-4161-9671-5c3cb67dc02e admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:25.741600 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52ea1eb-ed15-4161-9671-5c3cb67dc02e admin admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:25.742348 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52ea1eb-ed15-4161-9671-5c3cb67dc02e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:25.744465 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52ea1eb-ed15-4161-9671-5c3cb67dc02e admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:25.751671 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1403274-7b0f-441e-a6d5-e6c39a26b42a admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:25.759902 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632676b1-274d-4257-b518-79ab685c6751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:25.761666 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632676b1-274d-4257-b518-79ab685c6751 admin admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:25.762327 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632676b1-274d-4257-b518-79ab685c6751 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:25.762762 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632676b1-274d-4257-b518-79ab685c6751 admin admin] PATH_INFO: `/v3/projects/01edf84df6ed400e8ec3282c87eb8888/users/b49f064508cb481e820933549826b257/roles/66c1074780954f8e87ed11a5c4e8c36a` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:25.765463 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=87,keystone:INSERT=13 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:59:25.772274 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 789/1586] 10.222.0.64 () {60 vars in 1144 bytes} [Tue Aug 2 09:59:25 2022] POST /identity/v3/users => generated 288 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:59:25.773406 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52ea1eb-ed15-4161-9671-5c3cb67dc02e admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:25.788918 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632676b1-274d-4257-b518-79ab685c6751 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01edf84df6ed400e8ec3282c87eb8888, user_id=b49f064508cb481e820933549826b257, role_id=66c1074780954f8e87ed11a5c4e8c36a)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:25.791151 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52ea1eb-ed15-4161-9671-5c3cb67dc02e admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:25.793066 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632676b1-274d-4257-b518-79ab685c6751 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:25.797922 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe984ec-6087-47cd-aa80-d5a62e256a6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:25.798648 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe984ec-6087-47cd-aa80-d5a62e256a6d admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:25.799000 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe984ec-6087-47cd-aa80-d5a62e256a6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:25.799410 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe984ec-6087-47cd-aa80-d5a62e256a6d admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:25.800183 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe984ec-6087-47cd-aa80-d5a62e256a6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:25.804347 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe984ec-6087-47cd-aa80-d5a62e256a6d admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:25.810575 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 798/1587] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 09:59:25 2022] PUT /identity/v3/projects/01edf84df6ed400e8ec3282c87eb8888/users/b49f064508cb481e820933549826b257/roles/66c1074780954f8e87ed11a5c4e8c36a => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:59:25.811929 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632676b1-274d-4257-b518-79ab685c6751 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:25.819540 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 790/1588] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 09:59:25 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:59:25.820701 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe984ec-6087-47cd-aa80-d5a62e256a6d admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:25.832957 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe984ec-6087-47cd-aa80-d5a62e256a6d admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:25.907478 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632676b1-274d-4257-b518-79ab685c6751 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:25.913084 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1df8b7-f791-4083-855e-6bdca91021a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:25.913714 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1df8b7-f791-4083-855e-6bdca91021a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:25.914002 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1df8b7-f791-4083-855e-6bdca91021a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:25.914285 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1df8b7-f791-4083-855e-6bdca91021a7 admin admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:25.915186 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1df8b7-f791-4083-855e-6bdca91021a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:25.916108 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9245c0c-e8c6-4d77-a472-8705c6bb0a1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:25.917247 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1df8b7-f791-4083-855e-6bdca91021a7 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:25.917425 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9245c0c-e8c6-4d77-a472-8705c6bb0a1e admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:25.917812 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9245c0c-e8c6-4d77-a472-8705c6bb0a1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:25.918190 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9245c0c-e8c6-4d77-a472-8705c6bb0a1e admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:25.919126 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9245c0c-e8c6-4d77-a472-8705c6bb0a1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:25.921913 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9245c0c-e8c6-4d77-a472-8705c6bb0a1e admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:25.927578 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:59:25.940017 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 791/1589] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 09:59:25 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:59:25.943618 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9245c0c-e8c6-4d77-a472-8705c6bb0a1e admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:25.947166 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 799/1590] 10.222.0.64 () {60 vars in 1144 bytes} [Tue Aug 2 09:59:25 2022] POST /identity/v3/users => generated 292 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:59:25.948058 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1df8b7-f791-4083-855e-6bdca91021a7 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:25.959407 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9245c0c-e8c6-4d77-a472-8705c6bb0a1e admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:25.961306 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1df8b7-f791-4083-855e-6bdca91021a7 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:25.964230 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77a4719-327e-41ba-8661-960159eccc39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:25.964919 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77a4719-327e-41ba-8661-960159eccc39 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:25.965155 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77a4719-327e-41ba-8661-960159eccc39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:25.965431 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77a4719-327e-41ba-8661-960159eccc39 admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:25.966107 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b798dc54-76f1-452d-b38d-b1dd7b8db2a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:25.966265 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77a4719-327e-41ba-8661-960159eccc39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:25.966731 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b798dc54-76f1-452d-b38d-b1dd7b8db2a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:25.967021 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b798dc54-76f1-452d-b38d-b1dd7b8db2a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:25.967361 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b798dc54-76f1-452d-b38d-b1dd7b8db2a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:25.968602 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b798dc54-76f1-452d-b38d-b1dd7b8db2a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:25.968746 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77a4719-327e-41ba-8661-960159eccc39 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:25.970538 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b798dc54-76f1-452d-b38d-b1dd7b8db2a8 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:25.982410 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 800/1591] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 09:59:25 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:59:25.985030 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 792/1592] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 09:59:25 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:59:25.986263 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77a4719-327e-41ba-8661-960159eccc39 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:25.986873 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e05f58-8cbf-4754-84bd-6295dff3a692 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:25.987179 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e05f58-8cbf-4754-84bd-6295dff3a692 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:25.987464 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e05f58-8cbf-4754-84bd-6295dff3a692 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:25.997110 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77a4719-327e-41ba-8661-960159eccc39 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:26.002457 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454038ef-cded-4a56-ae35-d76ac18ff7b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:26.003075 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454038ef-cded-4a56-ae35-d76ac18ff7b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:26.003408 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454038ef-cded-4a56-ae35-d76ac18ff7b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:26.003695 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454038ef-cded-4a56-ae35-d76ac18ff7b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:26.004476 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454038ef-cded-4a56-ae35-d76ac18ff7b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:26.006795 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454038ef-cded-4a56-ae35-d76ac18ff7b7 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:26.008057 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-f0e05f58-8cbf-4754-84bd-6295dff3a692 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:59:26.009627 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-f0e05f58-8cbf-4754-84bd-6295dff3a692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:26.019564 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 801/1593] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 09:59:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:59:26.021606 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 793/1594] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 09:59:25 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:59:26.023363 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e05f58-8cbf-4754-84bd-6295dff3a692 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:26.025171 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454038ef-cded-4a56-ae35-d76ac18ff7b7 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:26.035999 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e05f58-8cbf-4754-84bd-6295dff3a692 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:26.037374 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454038ef-cded-4a56-ae35-d76ac18ff7b7 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:26.041827 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b045b2f-f2c8-4cab-8356-25c1ac8370e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:26.042461 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b045b2f-f2c8-4cab-8356-25c1ac8370e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:26.042582 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07bca9b-0ccf-41a9-b6d4-5d9542a1d576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:26.042747 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b045b2f-f2c8-4cab-8356-25c1ac8370e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:26.043030 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b045b2f-f2c8-4cab-8356-25c1ac8370e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:26.043240 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07bca9b-0ccf-41a9-b6d4-5d9542a1d576 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:26.043524 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07bca9b-0ccf-41a9-b6d4-5d9542a1d576 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:26.043797 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b045b2f-f2c8-4cab-8356-25c1ac8370e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:26.043912 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07bca9b-0ccf-41a9-b6d4-5d9542a1d576 admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:26.044645 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07bca9b-0ccf-41a9-b6d4-5d9542a1d576 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:26.045503 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b045b2f-f2c8-4cab-8356-25c1ac8370e0 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:26.046785 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07bca9b-0ccf-41a9-b6d4-5d9542a1d576 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:26.058587 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 802/1595] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 09:59:26 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:59:26.059458 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b045b2f-f2c8-4cab-8356-25c1ac8370e0 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:26.059587 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 794/1596] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 09:59:26 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:59:26.060469 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07bca9b-0ccf-41a9-b6d4-5d9542a1d576 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:26.074315 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07bca9b-0ccf-41a9-b6d4-5d9542a1d576 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:26.080282 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907bbed3-b465-4133-9122-d923b20da637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:26.080525 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b045b2f-f2c8-4cab-8356-25c1ac8370e0 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:26.080892 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907bbed3-b465-4133-9122-d923b20da637 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:26.081360 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907bbed3-b465-4133-9122-d923b20da637 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:26.081637 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907bbed3-b465-4133-9122-d923b20da637 admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:26.082341 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907bbed3-b465-4133-9122-d923b20da637 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:26.084542 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907bbed3-b465-4133-9122-d923b20da637 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:26.086608 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680cce82-d130-47bd-b47e-331ca6c00617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:26.088362 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680cce82-d130-47bd-b47e-331ca6c00617 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:26.088862 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680cce82-d130-47bd-b47e-331ca6c00617 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:26.089241 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680cce82-d130-47bd-b47e-331ca6c00617 admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:26.090059 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680cce82-d130-47bd-b47e-331ca6c00617 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:26.093254 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680cce82-d130-47bd-b47e-331ca6c00617 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:26.097780 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 795/1597] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 09:59:26 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:59:26.098844 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907bbed3-b465-4133-9122-d923b20da637 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:26.106926 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 803/1598] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 09:59:26 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:59:26.108883 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680cce82-d130-47bd-b47e-331ca6c00617 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:26.113122 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907bbed3-b465-4133-9122-d923b20da637 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:26.119839 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e486af-db77-45c2-b6e8-d489b13724bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:26.121095 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e486af-db77-45c2-b6e8-d489b13724bb admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:26.121373 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e486af-db77-45c2-b6e8-d489b13724bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:26.121654 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e486af-db77-45c2-b6e8-d489b13724bb admin admin] PATH_INFO: `/v3/projects/2cfcda91b1d24ae4bb380d4bae0ef090/users/134202fa63914e9ea9fa73655f4a8f67/roles/66c1074780954f8e87ed11a5c4e8c36a` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:26.123079 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680cce82-d130-47bd-b47e-331ca6c00617 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:26.129714 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036c5709-9c8d-4bcd-8395-525dd651cb80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:26.130347 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036c5709-9c8d-4bcd-8395-525dd651cb80 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:26.130634 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036c5709-9c8d-4bcd-8395-525dd651cb80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:26.131003 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036c5709-9c8d-4bcd-8395-525dd651cb80 admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:26.131589 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036c5709-9c8d-4bcd-8395-525dd651cb80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:26.134138 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036c5709-9c8d-4bcd-8395-525dd651cb80 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:26.143813 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e486af-db77-45c2-b6e8-d489b13724bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=2cfcda91b1d24ae4bb380d4bae0ef090, user_id=134202fa63914e9ea9fa73655f4a8f67, role_id=66c1074780954f8e87ed11a5c4e8c36a)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:26.146984 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e486af-db77-45c2-b6e8-d489b13724bb admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:26.149693 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 804/1599] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 09:59:26 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:59:26.150725 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036c5709-9c8d-4bcd-8395-525dd651cb80 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:26.152547 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-036c5709-9c8d-4bcd-8395-525dd651cb80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:26.172055 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 796/1600] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 09:59:26 2022] PUT /identity/v3/projects/2cfcda91b1d24ae4bb380d4bae0ef090/users/134202fa63914e9ea9fa73655f4a8f67/roles/66c1074780954f8e87ed11a5c4e8c36a => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:59:26.173172 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e486af-db77-45c2-b6e8-d489b13724bb admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:26.222139 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036c5709-9c8d-4bcd-8395-525dd651cb80 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:26.266895 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e486af-db77-45c2-b6e8-d489b13724bb admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:26.278170 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad8e974-44fb-4ad2-ad90-9cd79d1e09f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:26.281145 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad8e974-44fb-4ad2-ad90-9cd79d1e09f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:26.281983 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad8e974-44fb-4ad2-ad90-9cd79d1e09f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:26.283378 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad8e974-44fb-4ad2-ad90-9cd79d1e09f9 admin admin] PATH_INFO: `/v3/projects/d0b63cdea5bf43ea8478b59c7154ee70/users/c94f607a834f43e7a0073fe27d265901/roles/66c1074780954f8e87ed11a5c4e8c36a` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:26.289140 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7168da8-0116-4fa6-8537-d473eb6c0b04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:26.290035 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7168da8-0116-4fa6-8537-d473eb6c0b04 service neutron] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:26.290428 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7168da8-0116-4fa6-8537-d473eb6c0b04 service neutron] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:26.290849 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7168da8-0116-4fa6-8537-d473eb6c0b04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:26.304487 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7168da8-0116-4fa6-8537-d473eb6c0b04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:26.304788 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad8e974-44fb-4ad2-ad90-9cd79d1e09f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0b63cdea5bf43ea8478b59c7154ee70, user_id=c94f607a834f43e7a0073fe27d265901, role_id=66c1074780954f8e87ed11a5c4e8c36a)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:26.306454 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7168da8-0116-4fa6-8537-d473eb6c0b04 service neutron] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:26.309572 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad8e974-44fb-4ad2-ad90-9cd79d1e09f9 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:26.318089 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 805/1601] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 09:59:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:59:26.323741 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:59:26.331812 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7168da8-0116-4fa6-8537-d473eb6c0b04 service neutron] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:26.340679 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 797/1602] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 09:59:26 2022] PUT /identity/v3/projects/d0b63cdea5bf43ea8478b59c7154ee70/users/c94f607a834f43e7a0073fe27d265901/roles/66c1074780954f8e87ed11a5c4e8c36a => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:59:26.341595 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad8e974-44fb-4ad2-ad90-9cd79d1e09f9 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:26.358795 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7168da8-0116-4fa6-8537-d473eb6c0b04 service neutron] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:26.484414 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aacb244-87fc-40b5-b14e-3102c838c0a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:26.485605 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aacb244-87fc-40b5-b14e-3102c838c0a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:26.485729 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad8e974-44fb-4ad2-ad90-9cd79d1e09f9 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:26.485845 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aacb244-87fc-40b5-b14e-3102c838c0a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:26.486211 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aacb244-87fc-40b5-b14e-3102c838c0a7 admin admin] PATH_INFO: `/v3/projects/d1f5fe53a76440268bbb5b72e551efd9/users/dfe32e33711e42118f3ff5021762604f/roles/66c1074780954f8e87ed11a5c4e8c36a` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:26.489673 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:59:26.498184 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e556bb2-5872-4014-986c-8bf67626c85d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:26.499473 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e556bb2-5872-4014-986c-8bf67626c85d admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:26.499781 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e556bb2-5872-4014-986c-8bf67626c85d admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:26.500081 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e556bb2-5872-4014-986c-8bf67626c85d admin admin] PATH_INFO: `/v3/projects/dba5eb2dc9f549a5bf13a16b509dc3d4/users/982a01f0d3a5459a9add3f6f23db5db7/roles/66c1074780954f8e87ed11a5c4e8c36a` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:26.507457 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aacb244-87fc-40b5-b14e-3102c838c0a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1f5fe53a76440268bbb5b72e551efd9, user_id=dfe32e33711e42118f3ff5021762604f, role_id=66c1074780954f8e87ed11a5c4e8c36a)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:26.510598 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aacb244-87fc-40b5-b14e-3102c838c0a7 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:26.527003 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e556bb2-5872-4014-986c-8bf67626c85d admin admin] RBAC: Authorizing `identity:create_grant(project_id=dba5eb2dc9f549a5bf13a16b509dc3d4, user_id=982a01f0d3a5459a9add3f6f23db5db7, role_id=66c1074780954f8e87ed11a5c4e8c36a)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:26.536305 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e556bb2-5872-4014-986c-8bf67626c85d admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:26.546867 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 806/1603] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 09:59:26 2022] PUT /identity/v3/projects/d1f5fe53a76440268bbb5b72e551efd9/users/dfe32e33711e42118f3ff5021762604f/roles/66c1074780954f8e87ed11a5c4e8c36a => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:59:26.548513 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aacb244-87fc-40b5-b14e-3102c838c0a7 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:26.574350 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 798/1604] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 09:59:26 2022] PUT /identity/v3/projects/dba5eb2dc9f549a5bf13a16b509dc3d4/users/982a01f0d3a5459a9add3f6f23db5db7/roles/66c1074780954f8e87ed11a5c4e8c36a => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:59:26.576387 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e556bb2-5872-4014-986c-8bf67626c85d admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:26.654214 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aacb244-87fc-40b5-b14e-3102c838c0a7 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:26.660621 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e556bb2-5872-4014-986c-8bf67626c85d admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:26.661122 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdce257-1b85-4a68-bc22-e07536f42263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:26.663345 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdce257-1b85-4a68-bc22-e07536f42263 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:26.663641 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdce257-1b85-4a68-bc22-e07536f42263 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:26.663943 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdce257-1b85-4a68-bc22-e07536f42263 admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:26.664700 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdce257-1b85-4a68-bc22-e07536f42263 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:26.666953 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdce257-1b85-4a68-bc22-e07536f42263 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:26.667275 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962ae3eb-ffa5-4ad0-ab91-52e5e9c0d138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:26.668091 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962ae3eb-ffa5-4ad0-ab91-52e5e9c0d138 admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:26.668477 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962ae3eb-ffa5-4ad0-ab91-52e5e9c0d138 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:26.668855 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962ae3eb-ffa5-4ad0-ab91-52e5e9c0d138 admin admin] PATH_INFO: `/v3/projects/0db70d5950bd4f31bff4ea0ddbbe7810/users/1874b8463567457bb33f92fb0dbba126/roles/66c1074780954f8e87ed11a5c4e8c36a` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:26.683638 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 807/1605] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 09:59:26 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:59:26.683638 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca24b26-92a5-4ee9-ac65-5f52304c8c0c None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:26.683638 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca24b26-92a5-4ee9-ac65-5f52304c8c0c None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:26.683638 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca24b26-92a5-4ee9-ac65-5f52304c8c0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:26.691271 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962ae3eb-ffa5-4ad0-ab91-52e5e9c0d138 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0db70d5950bd4f31bff4ea0ddbbe7810, user_id=1874b8463567457bb33f92fb0dbba126, role_id=66c1074780954f8e87ed11a5c4e8c36a)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:26.699325 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962ae3eb-ffa5-4ad0-ab91-52e5e9c0d138 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:26.810060 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 799/1606] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 09:59:26 2022] PUT /identity/v3/projects/0db70d5950bd4f31bff4ea0ddbbe7810/users/1874b8463567457bb33f92fb0dbba126/roles/66c1074780954f8e87ed11a5c4e8c36a => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:59:26.810060 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd88d772-8947-456a-a06b-a945cc96ec3f None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:26.810060 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd88d772-8947-456a-a06b-a945cc96ec3f None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:26.810060 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd88d772-8947-456a-a06b-a945cc96ec3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:26.814528 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-9ca24b26-92a5-4ee9-ac65-5f52304c8c0c None None] MFA Rules not processed for user `134202fa63914e9ea9fa73655f4a8f67`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:59:26.814528 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-9ca24b26-92a5-4ee9-ac65-5f52304c8c0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:26.815093 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-cd88d772-8947-456a-a06b-a945cc96ec3f None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:59:26.815093 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-cd88d772-8947-456a-a06b-a945cc96ec3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:26.882235 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 808/1607] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 09:59:26 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:59:26.883634 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5632d3d-7878-4610-bbed-72be4cf5ab0f None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:26.883926 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5632d3d-7878-4610-bbed-72be4cf5ab0f None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:26.884205 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5632d3d-7878-4610-bbed-72be4cf5ab0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:26.932723 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-c5632d3d-7878-4610-bbed-72be4cf5ab0f None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:59:26.939815 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-c5632d3d-7878-4610-bbed-72be4cf5ab0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:26.947316 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 800/1608] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 09:59:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:59:26.948193 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c684cb53-b303-4ae0-806d-8d6646f0a4c8 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:26.948533 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c684cb53-b303-4ae0-806d-8d6646f0a4c8 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:26.948814 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c684cb53-b303-4ae0-806d-8d6646f0a4c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:26.964017 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 809/1609] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 09:59:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:59:26.965111 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5632d3d-7878-4610-bbed-72be4cf5ab0f None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:26.968081 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:59:26.974067 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-c684cb53-b303-4ae0-806d-8d6646f0a4c8 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:59:26.977107 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-c684cb53-b303-4ae0-806d-8d6646f0a4c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:26.978551 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5632d3d-7878-4610-bbed-72be4cf5ab0f None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:26.985027 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519b7a51-12c3-4bea-939f-b1aa58fc2265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:26.985642 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519b7a51-12c3-4bea-939f-b1aa58fc2265 admin admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:26.985931 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519b7a51-12c3-4bea-939f-b1aa58fc2265 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:26.986210 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519b7a51-12c3-4bea-939f-b1aa58fc2265 admin admin] PATH_INFO: `/v3/projects/dfc5594b6a9b4a118e1450ee0834003a/users/fad6fd5bc0d1405aaecbb79ccd43d8ce/roles/66c1074780954f8e87ed11a5c4e8c36a` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:26.991252 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 801/1610] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 09:59:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:59:26.993320 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3657e293-bfd7-45a9-8ea9-7bf74032cc22 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:26.993936 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3657e293-bfd7-45a9-8ea9-7bf74032cc22 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:26.994375 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3657e293-bfd7-45a9-8ea9-7bf74032cc22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:27.005954 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519b7a51-12c3-4bea-939f-b1aa58fc2265 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dfc5594b6a9b4a118e1450ee0834003a, user_id=fad6fd5bc0d1405aaecbb79ccd43d8ce, role_id=66c1074780954f8e87ed11a5c4e8c36a)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:27.008583 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519b7a51-12c3-4bea-939f-b1aa58fc2265 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:27.046593 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 810/1611] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 09:59:26 2022] PUT /identity/v3/projects/dfc5594b6a9b4a118e1450ee0834003a/users/fad6fd5bc0d1405aaecbb79ccd43d8ce/roles/66c1074780954f8e87ed11a5c4e8c36a => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:59:27.046593 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519b7a51-12c3-4bea-939f-b1aa58fc2265 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:27.068208 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-3657e293-bfd7-45a9-8ea9-7bf74032cc22 None None] MFA Rules not processed for user `1874b8463567457bb33f92fb0dbba126`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:59:27.069418 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-3657e293-bfd7-45a9-8ea9-7bf74032cc22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:27.148286 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:59:27.193330 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519b7a51-12c3-4bea-939f-b1aa58fc2265 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:27.210720 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77b4a3f-9fe9-4e91-a229-769b8aa5faf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:27.212715 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77b4a3f-9fe9-4e91-a229-769b8aa5faf3 admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:27.213168 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77b4a3f-9fe9-4e91-a229-769b8aa5faf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:27.213451 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77b4a3f-9fe9-4e91-a229-769b8aa5faf3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:27.218167 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77b4a3f-9fe9-4e91-a229-769b8aa5faf3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:27.221041 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77b4a3f-9fe9-4e91-a229-769b8aa5faf3 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:27.241215 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 802/1612] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 09:59:26 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:59:27.241902 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3657e293-bfd7-45a9-8ea9-7bf74032cc22 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:27.285506 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 811/1613] 10.222.0.64 () {60 vars in 1153 bytes} [Tue Aug 2 09:59:27 2022] POST /identity/v3/projects => generated 373 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:59:27.285506 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77b4a3f-9fe9-4e91-a229-769b8aa5faf3 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:27.375991 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3657e293-bfd7-45a9-8ea9-7bf74032cc22 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:27.378218 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77b4a3f-9fe9-4e91-a229-769b8aa5faf3 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:27.380137 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65a7ff4-eaa4-441e-b4a4-a3e6545d972b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:27.380857 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65a7ff4-eaa4-441e-b4a4-a3e6545d972b service neutron] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:27.381132 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65a7ff4-eaa4-441e-b4a4-a3e6545d972b service neutron] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:27.381471 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65a7ff4-eaa4-441e-b4a4-a3e6545d972b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:27.389437 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0316d7-e51a-42dc-afbf-fc26b1f62510 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:27.391130 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0316d7-e51a-42dc-afbf-fc26b1f62510 service neutron] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:27.391422 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0316d7-e51a-42dc-afbf-fc26b1f62510 service neutron] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:27.391703 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0316d7-e51a-42dc-afbf-fc26b1f62510 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:27.471934 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65a7ff4-eaa4-441e-b4a4-a3e6545d972b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:27.473790 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65a7ff4-eaa4-441e-b4a4-a3e6545d972b service neutron] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:27.489602 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 803/1614] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 09:59:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:59:27.491943 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081b1cef-32a8-48e3-b708-d2b2894e9ea6 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:27.492414 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081b1cef-32a8-48e3-b708-d2b2894e9ea6 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:27.492942 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081b1cef-32a8-48e3-b708-d2b2894e9ea6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:27.536897 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-081b1cef-32a8-48e3-b708-d2b2894e9ea6 None None] MFA Rules not processed for user `fad6fd5bc0d1405aaecbb79ccd43d8ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:59:27.538477 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-081b1cef-32a8-48e3-b708-d2b2894e9ea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:27.555821 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0316d7-e51a-42dc-afbf-fc26b1f62510 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:27.557678 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0316d7-e51a-42dc-afbf-fc26b1f62510 service neutron] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:27.571931 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 812/1615] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 09:59:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:59:27.573205 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0316d7-e51a-42dc-afbf-fc26b1f62510 service neutron] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:27.589950 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0316d7-e51a-42dc-afbf-fc26b1f62510 service neutron] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:27.603719 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02c5c6f-8d05-46df-9f1b-bae16f99f29b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:27.604484 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02c5c6f-8d05-46df-9f1b-bae16f99f29b service neutron] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:27.605896 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02c5c6f-8d05-46df-9f1b-bae16f99f29b service neutron] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:27.606705 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02c5c6f-8d05-46df-9f1b-bae16f99f29b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:27.616796 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:59:27.637049 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02c5c6f-8d05-46df-9f1b-bae16f99f29b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:27.640328 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02c5c6f-8d05-46df-9f1b-bae16f99f29b service neutron] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:27.671332 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 813/1616] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 09:59:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:59:27.672285 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02c5c6f-8d05-46df-9f1b-bae16f99f29b service neutron] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:27.700885 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 804/1617] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 09:59:27 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:59:27.702883 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc518ef2-2a18-4e70-a571-20e69c7ff23f None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:27.703186 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc518ef2-2a18-4e70-a571-20e69c7ff23f None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:27.703692 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc518ef2-2a18-4e70-a571-20e69c7ff23f None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:27.703828 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02c5c6f-8d05-46df-9f1b-bae16f99f29b service neutron] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:27.705223 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 805/1618] 10.222.0.64 () {54 vars in 962 bytes} [Tue Aug 2 09:59:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:59:27.705952 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc518ef2-2a18-4e70-a571-20e69c7ff23f None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:27.728184 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229ce9b3-57fd-4936-81ed-a10ab1edb9c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:27.729733 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229ce9b3-57fd-4936-81ed-a10ab1edb9c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:27.731166 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229ce9b3-57fd-4936-81ed-a10ab1edb9c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:27.731804 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229ce9b3-57fd-4936-81ed-a10ab1edb9c9 admin admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:27.734720 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229ce9b3-57fd-4936-81ed-a10ab1edb9c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:27.754283 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229ce9b3-57fd-4936-81ed-a10ab1edb9c9 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:27.768095 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc518ef2-2a18-4e70-a571-20e69c7ff23f None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:27.787101 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebec16f-7876-40ba-af93-cb34b5ddc98c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:27.788778 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebec16f-7876-40ba-af93-cb34b5ddc98c admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:27.789123 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebec16f-7876-40ba-af93-cb34b5ddc98c admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:27.789403 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebec16f-7876-40ba-af93-cb34b5ddc98c admin admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:27.796707 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebec16f-7876-40ba-af93-cb34b5ddc98c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:27.801329 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebec16f-7876-40ba-af93-cb34b5ddc98c admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:27.821918 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 814/1619] 10.222.0.64 () {60 vars in 1144 bytes} [Tue Aug 2 09:59:27 2022] POST /identity/v3/users => generated 293 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:59:27.825303 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6216a148-640c-4567-bd8d-75bee3413f7d None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:27.826384 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6216a148-640c-4567-bd8d-75bee3413f7d None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:27.826384 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6216a148-640c-4567-bd8d-75bee3413f7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:27.881059 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 806/1620] 10.222.0.64 () {60 vars in 1153 bytes} [Tue Aug 2 09:59:27 2022] POST /identity/v3/projects => generated 381 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:59:27.884452 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebec16f-7876-40ba-af93-cb34b5ddc98c admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:27.927827 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-6216a148-640c-4567-bd8d-75bee3413f7d None None] MFA Rules not processed for user `99655e1f9ca24472a86cd26b0dbd1fc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:59:27.931084 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-6216a148-640c-4567-bd8d-75bee3413f7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:27.994667 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:59:28.035758 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebec16f-7876-40ba-af93-cb34b5ddc98c admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:28.046677 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b436a64b-5f8a-41c9-9029-13b238f220bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:28.048745 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b436a64b-5f8a-41c9-9029-13b238f220bf admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:28.049313 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b436a64b-5f8a-41c9-9029-13b238f220bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:28.050332 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b436a64b-5f8a-41c9-9029-13b238f220bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:28.051247 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b436a64b-5f8a-41c9-9029-13b238f220bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:28.057109 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b436a64b-5f8a-41c9-9029-13b238f220bf admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:28.091762 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 807/1621] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 09:59:27 2022] GET /identity/v3/roles => generated 1420 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:59:28.091762 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b436a64b-5f8a-41c9-9029-13b238f220bf admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:28.119980 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 815/1622] 10.222.0.64 () {58 vars in 1059 bytes} [Tue Aug 2 09:59:27 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:59:28.125145 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6216a148-640c-4567-bd8d-75bee3413f7d None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:28.130784 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b436a64b-5f8a-41c9-9029-13b238f220bf admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:28.152202 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c450238-01d8-44f1-a838-5857585055d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:28.154112 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c450238-01d8-44f1-a838-5857585055d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:28.154402 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c450238-01d8-44f1-a838-5857585055d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:28.154718 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c450238-01d8-44f1-a838-5857585055d0 admin admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:28.155596 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c450238-01d8-44f1-a838-5857585055d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:28.157388 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c450238-01d8-44f1-a838-5857585055d0 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:28.168685 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6216a148-640c-4567-bd8d-75bee3413f7d None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:28.194567 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7d2c9c-411d-4258-8cdd-a0881fbfd341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:28.195487 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7d2c9c-411d-4258-8cdd-a0881fbfd341 admin admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:28.197238 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7d2c9c-411d-4258-8cdd-a0881fbfd341 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:28.197238 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7d2c9c-411d-4258-8cdd-a0881fbfd341 admin admin] PATH_INFO: `/v3/projects/844385826f9b4786b24f86575dbfb080/users/37325d549ec845a484f0b147c064d51d/roles/1000cf318fa14915b8091599eaead5de` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:28.244705 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 808/1623] 10.222.0.64 () {60 vars in 1144 bytes} [Tue Aug 2 09:59:28 2022] POST /identity/v3/users => generated 297 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:59:28.244705 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c450238-01d8-44f1-a838-5857585055d0 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:28.247698 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7d2c9c-411d-4258-8cdd-a0881fbfd341 admin admin] RBAC: Authorizing `identity:create_grant(project_id=844385826f9b4786b24f86575dbfb080, user_id=37325d549ec845a484f0b147c064d51d, role_id=1000cf318fa14915b8091599eaead5de)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:28.251836 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7d2c9c-411d-4258-8cdd-a0881fbfd341 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:28.268597 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c450238-01d8-44f1-a838-5857585055d0 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:28.275428 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440dd7d9-ddd6-4c4d-95c8-711a8c708339 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99655e1f9ca24472a86cd26b0dbd1fc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:28.276082 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440dd7d9-ddd6-4c4d-95c8-711a8c708339 service nova] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:28.288476 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 816/1624] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 09:59:28 2022] PUT /identity/v3/projects/844385826f9b4786b24f86575dbfb080/users/37325d549ec845a484f0b147c064d51d/roles/1000cf318fa14915b8091599eaead5de => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:59:28.288476 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7d2c9c-411d-4258-8cdd-a0881fbfd341 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:28.288780 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440dd7d9-ddd6-4c4d-95c8-711a8c708339 service nova] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:28.288780 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440dd7d9-ddd6-4c4d-95c8-711a8c708339 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:28.429319 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440dd7d9-ddd6-4c4d-95c8-711a8c708339 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:28.432657 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7d2c9c-411d-4258-8cdd-a0881fbfd341 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:28.433492 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440dd7d9-ddd6-4c4d-95c8-711a8c708339 service nova] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:28.445616 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3754ca7c-b9ab-4521-8187-7c9e1c51e2b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:28.446309 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3754ca7c-b9ab-4521-8187-7c9e1c51e2b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:28.446600 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3754ca7c-b9ab-4521-8187-7c9e1c51e2b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:28.446877 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3754ca7c-b9ab-4521-8187-7c9e1c51e2b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:28.447599 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3754ca7c-b9ab-4521-8187-7c9e1c51e2b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:28.450955 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3754ca7c-b9ab-4521-8187-7c9e1c51e2b0 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:28.456567 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 809/1625] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 09:59:28 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:59:28.457429 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440dd7d9-ddd6-4c4d-95c8-711a8c708339 service nova] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:28.476759 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 817/1626] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 09:59:28 2022] GET /identity/v3/roles => generated 1420 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:59:28.478755 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3754ca7c-b9ab-4521-8187-7c9e1c51e2b0 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:28.480280 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440dd7d9-ddd6-4c4d-95c8-711a8c708339 service nova] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:28.499111 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606acb2f-2884-4d2f-9ba2-5ae942fed3ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:28.499754 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606acb2f-2884-4d2f-9ba2-5ae942fed3ce admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:28.500043 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606acb2f-2884-4d2f-9ba2-5ae942fed3ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:28.500346 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606acb2f-2884-4d2f-9ba2-5ae942fed3ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:28.501060 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606acb2f-2884-4d2f-9ba2-5ae942fed3ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:28.503309 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3754ca7c-b9ab-4521-8187-7c9e1c51e2b0 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:28.506028 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606acb2f-2884-4d2f-9ba2-5ae942fed3ce admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:28.518891 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0907e1b1-8acd-4d5a-a930-b1ad3361c740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:28.519594 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0907e1b1-8acd-4d5a-a930-b1ad3361c740 admin admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:28.519881 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0907e1b1-8acd-4d5a-a930-b1ad3361c740 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:28.520160 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0907e1b1-8acd-4d5a-a930-b1ad3361c740 admin admin] PATH_INFO: `/v3/projects/6b0cf8a7576845d39063552bf712ef25/users/e1c4cd72c15d48139498b3dfa8b74241/roles/1000cf318fa14915b8091599eaead5de` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:28.526303 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 810/1627] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 09:59:28 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:59:28.530559 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606acb2f-2884-4d2f-9ba2-5ae942fed3ce admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:28.557162 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606acb2f-2884-4d2f-9ba2-5ae942fed3ce admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:28.558410 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0907e1b1-8acd-4d5a-a930-b1ad3361c740 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b0cf8a7576845d39063552bf712ef25, user_id=e1c4cd72c15d48139498b3dfa8b74241, role_id=1000cf318fa14915b8091599eaead5de)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:28.565647 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0907e1b1-8acd-4d5a-a930-b1ad3361c740 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:28.570492 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2055bb-8e39-48cd-ac46-e04e32b99374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:28.572108 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2055bb-8e39-48cd-ac46-e04e32b99374 admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:28.574459 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2055bb-8e39-48cd-ac46-e04e32b99374 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:28.574821 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2055bb-8e39-48cd-ac46-e04e32b99374 admin admin] PATH_INFO: `/v3/projects/844385826f9b4786b24f86575dbfb080/users/37325d549ec845a484f0b147c064d51d/roles/1000cf318fa14915b8091599eaead5de` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:28.579972 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2055bb-8e39-48cd-ac46-e04e32b99374 admin admin] RBAC: Authorizing `identity:create_grant(project_id=844385826f9b4786b24f86575dbfb080, user_id=37325d549ec845a484f0b147c064d51d, role_id=1000cf318fa14915b8091599eaead5de)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:28.585716 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2055bb-8e39-48cd-ac46-e04e32b99374 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:28.605637 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 818/1628] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 09:59:28 2022] PUT /identity/v3/projects/6b0cf8a7576845d39063552bf712ef25/users/e1c4cd72c15d48139498b3dfa8b74241/roles/1000cf318fa14915b8091599eaead5de => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:59:28.605758 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:59:28.606646 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0907e1b1-8acd-4d5a-a930-b1ad3361c740 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:28.628420 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 811/1629] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 09:59:28 2022] PUT /identity/v3/projects/844385826f9b4786b24f86575dbfb080/users/37325d549ec845a484f0b147c064d51d/roles/1000cf318fa14915b8091599eaead5de => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:59:28.630713 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e0dbd4-0241-4b60-a7c6-b28ad5329721 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:28.631062 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e0dbd4-0241-4b60-a7c6-b28ad5329721 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:28.631352 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e0dbd4-0241-4b60-a7c6-b28ad5329721 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:28.708543 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-77e0dbd4-0241-4b60-a7c6-b28ad5329721 None None] MFA Rules not processed for user `37325d549ec845a484f0b147c064d51d`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:59:28.712577 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-77e0dbd4-0241-4b60-a7c6-b28ad5329721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:28.765519 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0907e1b1-8acd-4d5a-a930-b1ad3361c740 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:28.801331 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:59:28.933458 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 812/1630] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 09:59:28 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:59:28.952221 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e0dbd4-0241-4b60-a7c6-b28ad5329721 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:28.966197 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38ff667-84b3-4a37-b1c2-36dee344d712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:28.968276 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38ff667-84b3-4a37-b1c2-36dee344d712 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:28.972235 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38ff667-84b3-4a37-b1c2-36dee344d712 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:28.972850 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38ff667-84b3-4a37-b1c2-36dee344d712 admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:28.975181 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38ff667-84b3-4a37-b1c2-36dee344d712 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:28.977097 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38ff667-84b3-4a37-b1c2-36dee344d712 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:29.028682 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 819/1631] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 09:59:28 2022] GET /identity/v3/roles => generated 1420 bytes in 422 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 09:59:29.030901 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38ff667-84b3-4a37-b1c2-36dee344d712 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:29.055041 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38ff667-84b3-4a37-b1c2-36dee344d712 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:29.070058 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4fa671-65ea-4b3e-8cb6-91e208280910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:29.071023 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4fa671-65ea-4b3e-8cb6-91e208280910 admin admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:29.071419 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4fa671-65ea-4b3e-8cb6-91e208280910 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:29.071823 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4fa671-65ea-4b3e-8cb6-91e208280910 admin admin] PATH_INFO: `/v3/projects/6b0cf8a7576845d39063552bf712ef25/users/e1c4cd72c15d48139498b3dfa8b74241/roles/1000cf318fa14915b8091599eaead5de` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:29.073657 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e0dbd4-0241-4b60-a7c6-b28ad5329721 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:29.074642 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4fa671-65ea-4b3e-8cb6-91e208280910 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b0cf8a7576845d39063552bf712ef25, user_id=e1c4cd72c15d48139498b3dfa8b74241, role_id=1000cf318fa14915b8091599eaead5de)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:29.078600 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4fa671-65ea-4b3e-8cb6-91e208280910 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:29.084792 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30be86b5-86cb-47e6-b0f6-9ab825275a52 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d7de62b9ee4039bdc79627ac7fb499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:29.085588 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30be86b5-86cb-47e6-b0f6-9ab825275a52 service cinder] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:29.085880 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30be86b5-86cb-47e6-b0f6-9ab825275a52 service cinder] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:29.087327 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30be86b5-86cb-47e6-b0f6-9ab825275a52 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:29.102568 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 820/1632] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 09:59:29 2022] PUT /identity/v3/projects/6b0cf8a7576845d39063552bf712ef25/users/e1c4cd72c15d48139498b3dfa8b74241/roles/1000cf318fa14915b8091599eaead5de => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:59:29.104772 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b177593-e7e7-48f4-8e6b-399bd495f5ef None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:29.105175 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b177593-e7e7-48f4-8e6b-399bd495f5ef None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:29.105709 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b177593-e7e7-48f4-8e6b-399bd495f5ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:29.113575 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30be86b5-86cb-47e6-b0f6-9ab825275a52 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:29.115946 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30be86b5-86cb-47e6-b0f6-9ab825275a52 service cinder] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:29.142707 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 813/1633] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 09:59:28 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:59:29.150032 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-6b177593-e7e7-48f4-8e6b-399bd495f5ef None None] MFA Rules not processed for user `e1c4cd72c15d48139498b3dfa8b74241`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:59:29.153539 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-6b177593-e7e7-48f4-8e6b-399bd495f5ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:29.374857 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 821/1634] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 09:59:29 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:59:29.391746 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30be86b5-86cb-47e6-b0f6-9ab825275a52 service cinder] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:29.408847 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30be86b5-86cb-47e6-b0f6-9ab825275a52 service cinder] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:29.569051 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c829253e-f98d-4ae8-83a9-43506db01574 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d7de62b9ee4039bdc79627ac7fb499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:29.570005 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c829253e-f98d-4ae8-83a9-43506db01574 service cinder] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:29.570193 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c829253e-f98d-4ae8-83a9-43506db01574 service cinder] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:29.570469 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c829253e-f98d-4ae8-83a9-43506db01574 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:29.600746 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c829253e-f98d-4ae8-83a9-43506db01574 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:29.602201 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c829253e-f98d-4ae8-83a9-43506db01574 service cinder] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:29.624925 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bae538-32d5-4bcd-bdf9-245e2e62ed54 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:29.624925 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bae538-32d5-4bcd-bdf9-245e2e62ed54 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:29.624925 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bae538-32d5-4bcd-bdf9-245e2e62ed54 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:29.625303 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 822/1635] 10.222.0.64 () {54 vars in 962 bytes} [Tue Aug 2 09:59:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:59:29.627147 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 814/1636] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 09:59:29 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:59:29.655986 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6b27ec-2ec6-4e69-9666-912c8133eb23 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:29.656299 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6b27ec-2ec6-4e69-9666-912c8133eb23 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:29.656567 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6b27ec-2ec6-4e69-9666-912c8133eb23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:29.711337 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-1e6b27ec-2ec6-4e69-9666-912c8133eb23 None None] MFA Rules not processed for user `99655e1f9ca24472a86cd26b0dbd1fc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:59:29.713697 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-1e6b27ec-2ec6-4e69-9666-912c8133eb23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:29.944369 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c829253e-f98d-4ae8-83a9-43506db01574 service cinder] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:29.954173 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c829253e-f98d-4ae8-83a9-43506db01574 service cinder] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:29.980729 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e779244b-ea76-49d9-bee1-a86fe1dd4871 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d7de62b9ee4039bdc79627ac7fb499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:29.981372 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e779244b-ea76-49d9-bee1-a86fe1dd4871 service cinder] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:29.982111 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e779244b-ea76-49d9-bee1-a86fe1dd4871 service cinder] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:29.982111 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e779244b-ea76-49d9-bee1-a86fe1dd4871 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:30.006491 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 823/1637] 10.222.0.64 () {58 vars in 1059 bytes} [Tue Aug 2 09:59:29 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 351 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:59:30.017866 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcb898c-8b9d-48b5-8a2a-73c32d0290fe None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:30.018164 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcb898c-8b9d-48b5-8a2a-73c32d0290fe None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:30.018446 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcb898c-8b9d-48b5-8a2a-73c32d0290fe None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:30.022966 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 824/1638] 10.222.0.64 () {54 vars in 912 bytes} [Tue Aug 2 09:59:30 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:59:30.028583 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee0090e-e940-40f1-ac7c-2be172a6c39f None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:30.029030 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee0090e-e940-40f1-ac7c-2be172a6c39f None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:30.029629 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee0090e-e940-40f1-ac7c-2be172a6c39f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:30.069161 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:59:30.084161 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-aee0090e-e940-40f1-ac7c-2be172a6c39f None None] MFA Rules not processed for user `99655e1f9ca24472a86cd26b0dbd1fc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:59:30.086051 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-aee0090e-e940-40f1-ac7c-2be172a6c39f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:30.124154 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 825/1639] 10.222.0.64 () {58 vars in 1009 bytes} [Tue Aug 2 09:59:30 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:59:30.130759 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e779244b-ea76-49d9-bee1-a86fe1dd4871 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:30.136393 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e779244b-ea76-49d9-bee1-a86fe1dd4871 service cinder] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:30.143320 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee0090e-e940-40f1-ac7c-2be172a6c39f None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:30.153652 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee0090e-e940-40f1-ac7c-2be172a6c39f None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:30.158713 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81eb95c9-6c26-42fb-b36a-e2224e6fa567 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d7de62b9ee4039bdc79627ac7fb499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:30.159726 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81eb95c9-6c26-42fb-b36a-e2224e6fa567 service cinder] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:30.160221 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81eb95c9-6c26-42fb-b36a-e2224e6fa567 service cinder] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:30.160779 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81eb95c9-6c26-42fb-b36a-e2224e6fa567 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:30.169224 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 815/1640] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 09:59:29 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 09:59:30.172083 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:59:30.175544 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81eb95c9-6c26-42fb-b36a-e2224e6fa567 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:30.177355 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81eb95c9-6c26-42fb-b36a-e2224e6fa567 service cinder] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:30.201133 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 826/1641] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 09:59:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 09:59:30.439090 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319218a2-8986-4831-b407-847c036c95f4 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:30.439293 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319218a2-8986-4831-b407-847c036c95f4 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:30.439567 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319218a2-8986-4831-b407-847c036c95f4 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:30.448054 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 816/1642] 10.222.0.64 () {54 vars in 912 bytes} [Tue Aug 2 09:59:30 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:59:30.459490 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cd00c9-ff12-47b4-962a-21d75dc7cb1d None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:30.460306 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cd00c9-ff12-47b4-962a-21d75dc7cb1d None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:30.460970 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cd00c9-ff12-47b4-962a-21d75dc7cb1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:30.490369 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-75cd00c9-ff12-47b4-962a-21d75dc7cb1d None None] MFA Rules not processed for user `99655e1f9ca24472a86cd26b0dbd1fc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:59:30.492085 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-75cd00c9-ff12-47b4-962a-21d75dc7cb1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:30.506603 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 827/1643] 10.222.0.64 () {58 vars in 1009 bytes} [Tue Aug 2 09:59:30 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:59:30.516641 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319218a2-8986-4831-b407-847c036c95f4 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:30.524077 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cd00c9-ff12-47b4-962a-21d75dc7cb1d None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:30.552585 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cd00c9-ff12-47b4-962a-21d75dc7cb1d None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:30.566556 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630c0d96-d62f-4113-9195-634a5ce13ce4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d7de62b9ee4039bdc79627ac7fb499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:30.567198 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630c0d96-d62f-4113-9195-634a5ce13ce4 service cinder] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:30.567485 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630c0d96-d62f-4113-9195-634a5ce13ce4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:30.567764 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630c0d96-d62f-4113-9195-634a5ce13ce4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:30.611654 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630c0d96-d62f-4113-9195-634a5ce13ce4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:30.613403 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630c0d96-d62f-4113-9195-634a5ce13ce4 service cinder] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:30.647081 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 828/1644] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 09:59:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 09:59:30.718901 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319218a2-8986-4831-b407-847c036c95f4 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:30.731667 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8491be2c-b415-4443-bc72-8924c62287ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:30.733104 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8491be2c-b415-4443-bc72-8924c62287ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:30.734897 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8491be2c-b415-4443-bc72-8924c62287ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:30.792431 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8491be2c-b415-4443-bc72-8924c62287ca admin admin] PATH_INFO: `/v3/users/134202fa63914e9ea9fa73655f4a8f67` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:30.792431 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8491be2c-b415-4443-bc72-8924c62287ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=134202fa63914e9ea9fa73655f4a8f67)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:30.792431 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8491be2c-b415-4443-bc72-8924c62287ca admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:30.807680 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630c0d96-d62f-4113-9195-634a5ce13ce4 service cinder] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:30.829726 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630c0d96-d62f-4113-9195-634a5ce13ce4 service cinder] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:30.842429 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b325b09f-490d-430c-892b-a176b449e638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:30.843344 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b325b09f-490d-430c-892b-a176b449e638 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:30.843758 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b325b09f-490d-430c-892b-a176b449e638 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:30.844178 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b325b09f-490d-430c-892b-a176b449e638 admin admin] PATH_INFO: `/v3/users/1874b8463567457bb33f92fb0dbba126` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:30.849223 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b325b09f-490d-430c-892b-a176b449e638 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1874b8463567457bb33f92fb0dbba126)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:30.850470 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: WARNING py.warnings [None req-b325b09f-490d-430c-892b-a176b449e638 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 02 09:59:30.850470 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: warnings.warn(msg) Aug 02 09:59:30.850470 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: Aug 02 09:59:30.852387 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b325b09f-490d-430c-892b-a176b449e638 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:30.901916 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-8491be2c-b415-4443-bc72-8924c62287ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '134202fa63914e9ea9fa73655f4a8f67'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:59:30.917526 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-8491be2c-b415-4443-bc72-8924c62287ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '134202fa63914e9ea9fa73655f4a8f67'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:59:30.920588 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-8491be2c-b415-4443-bc72-8924c62287ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '134202fa63914e9ea9fa73655f4a8f67'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:59:30.945652 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-8491be2c-b415-4443-bc72-8924c62287ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '134202fa63914e9ea9fa73655f4a8f67'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:59:30.985403 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-8491be2c-b415-4443-bc72-8924c62287ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '134202fa63914e9ea9fa73655f4a8f67'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:59:31.002408 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 817/1645] 10.222.0.64 () {58 vars in 1224 bytes} [Tue Aug 2 09:59:30 2022] DELETE /identity/v3/users/134202fa63914e9ea9fa73655f4a8f67 => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:59:31.006652 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134b0e44-a4f1-4196-ac8e-f342aa5bb027 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:31.007411 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134b0e44-a4f1-4196-ac8e-f342aa5bb027 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:31.008066 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134b0e44-a4f1-4196-ac8e-f342aa5bb027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:31.012861 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-b325b09f-490d-430c-892b-a176b449e638 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1874b8463567457bb33f92fb0dbba126'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:59:31.033885 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-b325b09f-490d-430c-892b-a176b449e638 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1874b8463567457bb33f92fb0dbba126'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:59:31.047187 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-134b0e44-a4f1-4196-ac8e-f342aa5bb027 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:59:31.047368 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-b325b09f-490d-430c-892b-a176b449e638 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1874b8463567457bb33f92fb0dbba126'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:59:31.048169 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-b325b09f-490d-430c-892b-a176b449e638 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1874b8463567457bb33f92fb0dbba126'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:59:31.049003 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-b325b09f-490d-430c-892b-a176b449e638 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1874b8463567457bb33f92fb0dbba126'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:59:31.049532 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-134b0e44-a4f1-4196-ac8e-f342aa5bb027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:31.117741 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 829/1646] 10.222.0.64 () {58 vars in 1224 bytes} [Tue Aug 2 09:59:30 2022] DELETE /identity/v3/users/1874b8463567457bb33f92fb0dbba126 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:59:31.123921 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366bdabb-078e-4afc-baac-ff2b4a50e546 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:31.124528 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366bdabb-078e-4afc-baac-ff2b4a50e546 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:31.128962 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366bdabb-078e-4afc-baac-ff2b4a50e546 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:31.219811 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-366bdabb-078e-4afc-baac-ff2b4a50e546 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:59:31.222430 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-366bdabb-078e-4afc-baac-ff2b4a50e546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:31.278375 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:59:31.338557 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 818/1647] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 09:59:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:59:31.446244 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 830/1648] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 09:59:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:59:31.451303 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134b0e44-a4f1-4196-ac8e-f342aa5bb027 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:31.452920 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-134b0e44-a4f1-4196-ac8e-f342aa5bb027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:31.480955 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366bdabb-078e-4afc-baac-ff2b4a50e546 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:31.483835 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-366bdabb-078e-4afc-baac-ff2b4a50e546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:31.617336 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134b0e44-a4f1-4196-ac8e-f342aa5bb027 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:31.650661 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:59:31.664909 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366bdabb-078e-4afc-baac-ff2b4a50e546 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:31.673160 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3194e07e-f836-4ca0-acad-0cbe753c0a30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:31.679141 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3194e07e-f836-4ca0-acad-0cbe753c0a30 service neutron] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:31.679729 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3194e07e-f836-4ca0-acad-0cbe753c0a30 service neutron] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:31.680259 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3194e07e-f836-4ca0-acad-0cbe753c0a30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:31.703735 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3194e07e-f836-4ca0-acad-0cbe753c0a30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:31.705900 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3194e07e-f836-4ca0-acad-0cbe753c0a30 service neutron] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:31.727606 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 831/1649] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 09:59:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:59:31.732030 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ec31f4-811e-4c7b-8113-1ac53c6597d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:31.734222 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ec31f4-811e-4c7b-8113-1ac53c6597d3 service neutron] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:31.734833 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ec31f4-811e-4c7b-8113-1ac53c6597d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:31.735380 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ec31f4-811e-4c7b-8113-1ac53c6597d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:31.749319 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ec31f4-811e-4c7b-8113-1ac53c6597d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:31.753264 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ec31f4-811e-4c7b-8113-1ac53c6597d3 service neutron] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:31.781502 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 819/1650] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 09:59:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 09:59:33.621990 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3194e07e-f836-4ca0-acad-0cbe753c0a30 service neutron] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:33.625056 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-3194e07e-f836-4ca0-acad-0cbe753c0a30 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:33.655972 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3194e07e-f836-4ca0-acad-0cbe753c0a30 service neutron] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:33.681604 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d2ae07-afb0-4288-976c-dc2537f1c3a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:33.685793 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d2ae07-afb0-4288-976c-dc2537f1c3a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:33.687045 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d2ae07-afb0-4288-976c-dc2537f1c3a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:33.687527 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d2ae07-afb0-4288-976c-dc2537f1c3a6 admin admin] PATH_INFO: `/v3/projects/0db70d5950bd4f31bff4ea0ddbbe7810` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:33.691610 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d2ae07-afb0-4288-976c-dc2537f1c3a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0db70d5950bd4f31bff4ea0ddbbe7810)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:33.701231 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d2ae07-afb0-4288-976c-dc2537f1c3a6 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:33.840945 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-c1d2ae07-afb0-4288-976c-dc2537f1c3a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0db70d5950bd4f31bff4ea0ddbbe7810', 'request_id': 'req-c1d2ae07-afb0-4288-976c-dc2537f1c3a6'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:59:33.859074 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-c1d2ae07-afb0-4288-976c-dc2537f1c3a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0db70d5950bd4f31bff4ea0ddbbe7810', 'request_id': 'req-c1d2ae07-afb0-4288-976c-dc2537f1c3a6'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:59:33.872907 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-c1d2ae07-afb0-4288-976c-dc2537f1c3a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0db70d5950bd4f31bff4ea0ddbbe7810', 'request_id': 'req-c1d2ae07-afb0-4288-976c-dc2537f1c3a6'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:59:33.911029 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-c1d2ae07-afb0-4288-976c-dc2537f1c3a6 admin admin] The token cache is being invalidate because project 0db70d5950bd4f31bff4ea0ddbbe7810 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80153) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:59:33.913000 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-c1d2ae07-afb0-4288-976c-dc2537f1c3a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:59:33.915838 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-c1d2ae07-afb0-4288-976c-dc2537f1c3a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:59:33.918654 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ec31f4-811e-4c7b-8113-1ac53c6597d3 service neutron] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:33.922929 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-14ec31f4-811e-4c7b-8113-1ac53c6597d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:33.925102 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 832/1651] 10.222.0.64 () {58 vars in 1233 bytes} [Tue Aug 2 09:59:33 2022] DELETE /identity/v3/projects/0db70d5950bd4f31bff4ea0ddbbe7810 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:59:33.930152 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f7b433-176f-4f42-a9e4-77397c99716f None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:33.930455 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f7b433-176f-4f42-a9e4-77397c99716f None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:33.930792 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f7b433-176f-4f42-a9e4-77397c99716f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:33.955550 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ec31f4-811e-4c7b-8113-1ac53c6597d3 service neutron] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:33.967057 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-01f7b433-176f-4f42-a9e4-77397c99716f None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:59:33.968329 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-01f7b433-176f-4f42-a9e4-77397c99716f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:34.079543 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fed8110-edd2-4b77-ba03-a040e71694e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:34.081592 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fed8110-edd2-4b77-ba03-a040e71694e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:34.082099 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fed8110-edd2-4b77-ba03-a040e71694e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:34.082731 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fed8110-edd2-4b77-ba03-a040e71694e2 admin admin] PATH_INFO: `/v3/projects/2cfcda91b1d24ae4bb380d4bae0ef090` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:34.084194 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fed8110-edd2-4b77-ba03-a040e71694e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2cfcda91b1d24ae4bb380d4bae0ef090)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:34.086966 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: WARNING py.warnings [None req-7fed8110-edd2-4b77-ba03-a040e71694e2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 02 09:59:34.086966 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: warnings.warn(msg) Aug 02 09:59:34.086966 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: Aug 02 09:59:34.089621 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fed8110-edd2-4b77-ba03-a040e71694e2 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:34.124911 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 833/1652] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 09:59:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:59:34.132974 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f8e6e5-bc01-46e6-ac89-ac5e6a95b385 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:34.133449 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f8e6e5-bc01-46e6-ac89-ac5e6a95b385 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:34.134072 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f8e6e5-bc01-46e6-ac89-ac5e6a95b385 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:34.176491 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-36f8e6e5-bc01-46e6-ac89-ac5e6a95b385 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:59:34.185140 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-36f8e6e5-bc01-46e6-ac89-ac5e6a95b385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:34.219189 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-7fed8110-edd2-4b77-ba03-a040e71694e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2cfcda91b1d24ae4bb380d4bae0ef090', 'request_id': 'req-7fed8110-edd2-4b77-ba03-a040e71694e2'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:59:34.224402 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:59:34.237658 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-7fed8110-edd2-4b77-ba03-a040e71694e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2cfcda91b1d24ae4bb380d4bae0ef090', 'request_id': 'req-7fed8110-edd2-4b77-ba03-a040e71694e2'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:59:34.280951 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-7fed8110-edd2-4b77-ba03-a040e71694e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2cfcda91b1d24ae4bb380d4bae0ef090', 'request_id': 'req-7fed8110-edd2-4b77-ba03-a040e71694e2'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:59:34.319422 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-7fed8110-edd2-4b77-ba03-a040e71694e2 admin admin] The token cache is being invalidate because project 2cfcda91b1d24ae4bb380d4bae0ef090 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80154) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:59:34.319745 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-7fed8110-edd2-4b77-ba03-a040e71694e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:59:34.320453 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-7fed8110-edd2-4b77-ba03-a040e71694e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:59:34.326810 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 820/1653] 10.222.0.64 () {58 vars in 1233 bytes} [Tue Aug 2 09:59:33 2022] DELETE /identity/v3/projects/2cfcda91b1d24ae4bb380d4bae0ef090 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:59:34.328369 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fed8110-edd2-4b77-ba03-a040e71694e2 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:34.333695 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-7fed8110-edd2-4b77-ba03-a040e71694e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:34.372234 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 834/1654] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 09:59:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:59:34.379177 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f8e6e5-bc01-46e6-ac89-ac5e6a95b385 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:34.405446 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f8e6e5-bc01-46e6-ac89-ac5e6a95b385 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:34.421671 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4938d76-f4d5-44bc-b6be-e631f719e332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:34.423820 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4938d76-f4d5-44bc-b6be-e631f719e332 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:34.425898 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4938d76-f4d5-44bc-b6be-e631f719e332 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:34.426832 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4938d76-f4d5-44bc-b6be-e631f719e332 admin admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:34.427840 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4938d76-f4d5-44bc-b6be-e631f719e332 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:59:34.428163 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4938d76-f4d5-44bc-b6be-e631f719e332 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:34.435116 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4938d76-f4d5-44bc-b6be-e631f719e332 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:34.473272 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 835/1655] 10.222.0.64 () {58 vars in 1153 bytes} [Tue Aug 2 09:59:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:59:34.475239 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4938d76-f4d5-44bc-b6be-e631f719e332 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:34.496503 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:59:34.515512 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4938d76-f4d5-44bc-b6be-e631f719e332 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:34.539237 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0006b7-ec59-4110-944b-4c36ddc6f2b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:34.540039 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0006b7-ec59-4110-944b-4c36ddc6f2b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:34.540631 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0006b7-ec59-4110-944b-4c36ddc6f2b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:34.541214 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0006b7-ec59-4110-944b-4c36ddc6f2b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:34.542633 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fed8110-edd2-4b77-ba03-a040e71694e2 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:34.543183 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0006b7-ec59-4110-944b-4c36ddc6f2b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:34.548549 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0006b7-ec59-4110-944b-4c36ddc6f2b7 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:34.556082 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f07a20e-bbbf-42c2-89fb-6ab8cdaf04da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:34.559008 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f07a20e-bbbf-42c2-89fb-6ab8cdaf04da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:34.559291 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f07a20e-bbbf-42c2-89fb-6ab8cdaf04da admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:34.559585 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f07a20e-bbbf-42c2-89fb-6ab8cdaf04da admin admin] PATH_INFO: `/v3/users/37325d549ec845a484f0b147c064d51d` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:34.561358 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f07a20e-bbbf-42c2-89fb-6ab8cdaf04da admin admin] RBAC: Authorizing `identity:delete_user(user_id=37325d549ec845a484f0b147c064d51d)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:34.576891 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f07a20e-bbbf-42c2-89fb-6ab8cdaf04da admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:34.614203 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 836/1656] 10.222.0.64 () {60 vars in 1153 bytes} [Tue Aug 2 09:59:34 2022] POST /identity/v3/projects => generated 405 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:59:34.619797 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f3fe94-3edd-47e4-8946-bf58dbe131d1 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:34.620122 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f3fe94-3edd-47e4-8946-bf58dbe131d1 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:34.620790 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f3fe94-3edd-47e4-8946-bf58dbe131d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:34.693508 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-93f3fe94-3edd-47e4-8946-bf58dbe131d1 None None] MFA Rules not processed for user `5bbba5ae48cc4ec39467df736ad1bbad`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:59:34.695124 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-3f07a20e-bbbf-42c2-89fb-6ab8cdaf04da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '37325d549ec845a484f0b147c064d51d'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:59:34.711046 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-93f3fe94-3edd-47e4-8946-bf58dbe131d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:34.744297 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-3f07a20e-bbbf-42c2-89fb-6ab8cdaf04da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '37325d549ec845a484f0b147c064d51d'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:59:34.748886 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-3f07a20e-bbbf-42c2-89fb-6ab8cdaf04da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '37325d549ec845a484f0b147c064d51d'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:59:34.773617 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-3f07a20e-bbbf-42c2-89fb-6ab8cdaf04da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '37325d549ec845a484f0b147c064d51d'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:59:34.804921 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-3f07a20e-bbbf-42c2-89fb-6ab8cdaf04da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '37325d549ec845a484f0b147c064d51d'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:59:34.819822 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 821/1657] 10.222.0.64 () {58 vars in 1224 bytes} [Tue Aug 2 09:59:34 2022] DELETE /identity/v3/users/37325d549ec845a484f0b147c064d51d => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:59:34.821003 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f07a20e-bbbf-42c2-89fb-6ab8cdaf04da admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:34.826855 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-3f07a20e-bbbf-42c2-89fb-6ab8cdaf04da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:34.918438 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 837/1658] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 09:59:34 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:59:34.930455 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e194b141-1046-41af-abe6-6925425b9cd9 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:34.930762 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e194b141-1046-41af-abe6-6925425b9cd9 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:34.931053 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e194b141-1046-41af-abe6-6925425b9cd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:34.998436 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-e194b141-1046-41af-abe6-6925425b9cd9 None None] MFA Rules not processed for user `5bbba5ae48cc4ec39467df736ad1bbad`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:59:35.001761 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-e194b141-1046-41af-abe6-6925425b9cd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:35.020554 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:59:35.033865 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f07a20e-bbbf-42c2-89fb-6ab8cdaf04da admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:35.045754 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e71e8bb-7c6c-40d0-8a0b-c7f6aca7dbff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:35.049763 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e71e8bb-7c6c-40d0-8a0b-c7f6aca7dbff admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:35.050217 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e71e8bb-7c6c-40d0-8a0b-c7f6aca7dbff admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:35.050723 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e71e8bb-7c6c-40d0-8a0b-c7f6aca7dbff admin admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:35.051525 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e71e8bb-7c6c-40d0-8a0b-c7f6aca7dbff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:35.053668 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e71e8bb-7c6c-40d0-8a0b-c7f6aca7dbff admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:35.117587 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 822/1659] 10.222.0.64 () {60 vars in 1144 bytes} [Tue Aug 2 09:59:34 2022] POST /identity/v3/users => generated 303 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:59:35.118861 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e71e8bb-7c6c-40d0-8a0b-c7f6aca7dbff admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:35.139923 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e71e8bb-7c6c-40d0-8a0b-c7f6aca7dbff admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:35.154709 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e721a4-60e3-42d8-9cd6-42a4f778ea3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:35.158555 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 838/1660] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 09:59:34 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:59:35.163216 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e721a4-60e3-42d8-9cd6-42a4f778ea3d admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:35.163216 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e721a4-60e3-42d8-9cd6-42a4f778ea3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:35.163216 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e721a4-60e3-42d8-9cd6-42a4f778ea3d admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:35.163216 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e721a4-60e3-42d8-9cd6-42a4f778ea3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:35.163216 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e721a4-60e3-42d8-9cd6-42a4f778ea3d admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:35.171196 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e194b141-1046-41af-abe6-6925425b9cd9 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:35.172289 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-e194b141-1046-41af-abe6-6925425b9cd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:35.193522 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 823/1661] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 09:59:35 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:59:35.194760 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e721a4-60e3-42d8-9cd6-42a4f778ea3d admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:35.222091 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e721a4-60e3-42d8-9cd6-42a4f778ea3d admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:35.237353 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070081e7-f2f5-48be-8b85-29438fad04d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:35.238241 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070081e7-f2f5-48be-8b85-29438fad04d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:35.238671 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070081e7-f2f5-48be-8b85-29438fad04d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:35.239157 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070081e7-f2f5-48be-8b85-29438fad04d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:35.240023 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070081e7-f2f5-48be-8b85-29438fad04d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:35.241862 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070081e7-f2f5-48be-8b85-29438fad04d4 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:35.276751 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 824/1662] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 09:59:35 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:59:35.277885 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070081e7-f2f5-48be-8b85-29438fad04d4 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:35.290216 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e194b141-1046-41af-abe6-6925425b9cd9 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:35.297839 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070081e7-f2f5-48be-8b85-29438fad04d4 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:35.298690 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f933b448-aa85-44d2-9471-93e8568e9862 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99655e1f9ca24472a86cd26b0dbd1fc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:35.299622 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f933b448-aa85-44d2-9471-93e8568e9862 service nova] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:35.300094 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f933b448-aa85-44d2-9471-93e8568e9862 service nova] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:35.300608 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f933b448-aa85-44d2-9471-93e8568e9862 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:35.309218 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6a79a0-081d-4169-99dc-39ee372edf17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:35.310209 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6a79a0-081d-4169-99dc-39ee372edf17 admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:35.310428 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6a79a0-081d-4169-99dc-39ee372edf17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:35.311263 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6a79a0-081d-4169-99dc-39ee372edf17 admin admin] PATH_INFO: `/v3/projects/463e77fb077b4c26a9f6ff5a19b4d7ee/users/b663b479be0b4ae1b9ba2b79bd6872cb/roles/66c1074780954f8e87ed11a5c4e8c36a` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:35.314179 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f933b448-aa85-44d2-9471-93e8568e9862 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:35.316650 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f933b448-aa85-44d2-9471-93e8568e9862 service nova] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:35.323297 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:59:35.335245 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 839/1663] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 09:59:35 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:59:35.356708 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6a79a0-081d-4169-99dc-39ee372edf17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=463e77fb077b4c26a9f6ff5a19b4d7ee, user_id=b663b479be0b4ae1b9ba2b79bd6872cb, role_id=66c1074780954f8e87ed11a5c4e8c36a)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:35.359574 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6a79a0-081d-4169-99dc-39ee372edf17 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:35.401937 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 825/1664] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 09:59:35 2022] PUT /identity/v3/projects/463e77fb077b4c26a9f6ff5a19b4d7ee/users/b663b479be0b4ae1b9ba2b79bd6872cb/roles/66c1074780954f8e87ed11a5c4e8c36a => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:59:35.413648 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505e587b-2c92-4215-9a3c-4896509d9b44 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:35.413963 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505e587b-2c92-4215-9a3c-4896509d9b44 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:35.414283 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505e587b-2c92-4215-9a3c-4896509d9b44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:35.468787 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-505e587b-2c92-4215-9a3c-4896509d9b44 None None] MFA Rules not processed for user `b663b479be0b4ae1b9ba2b79bd6872cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:59:35.474372 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-505e587b-2c92-4215-9a3c-4896509d9b44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:35.572364 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 840/1665] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 09:59:35 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:59:35.573824 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6a79a0-081d-4169-99dc-39ee372edf17 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:35.628943 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505e587b-2c92-4215-9a3c-4896509d9b44 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:35.630590 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-505e587b-2c92-4215-9a3c-4896509d9b44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:35.730413 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6a79a0-081d-4169-99dc-39ee372edf17 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:35.730926 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505e587b-2c92-4215-9a3c-4896509d9b44 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:35.742723 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698f134f-1ef4-4168-acd2-5bb6b3e58fa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:35.743407 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698f134f-1ef4-4168-acd2-5bb6b3e58fa1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:35.743698 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698f134f-1ef4-4168-acd2-5bb6b3e58fa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:35.744151 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698f134f-1ef4-4168-acd2-5bb6b3e58fa1 admin admin] PATH_INFO: `/v3/users/fad6fd5bc0d1405aaecbb79ccd43d8ce` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:35.745401 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698f134f-1ef4-4168-acd2-5bb6b3e58fa1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fad6fd5bc0d1405aaecbb79ccd43d8ce)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:35.747363 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698f134f-1ef4-4168-acd2-5bb6b3e58fa1 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:35.751297 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3a0e6f-8e0b-4418-9034-d5c6097269d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:35.751927 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3a0e6f-8e0b-4418-9034-d5c6097269d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:35.752221 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3a0e6f-8e0b-4418-9034-d5c6097269d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:35.752575 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3a0e6f-8e0b-4418-9034-d5c6097269d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:35.753263 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3a0e6f-8e0b-4418-9034-d5c6097269d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:35.755191 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3a0e6f-8e0b-4418-9034-d5c6097269d9 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:35.812007 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 826/1666] 10.222.0.64 () {60 vars in 1153 bytes} [Tue Aug 2 09:59:35 2022] POST /identity/v3/projects => generated 405 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:59:35.815971 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3a0e6f-8e0b-4418-9034-d5c6097269d9 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:35.886451 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-698f134f-1ef4-4168-acd2-5bb6b3e58fa1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fad6fd5bc0d1405aaecbb79ccd43d8ce'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:59:35.903742 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-698f134f-1ef4-4168-acd2-5bb6b3e58fa1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fad6fd5bc0d1405aaecbb79ccd43d8ce'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:59:35.917418 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-698f134f-1ef4-4168-acd2-5bb6b3e58fa1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fad6fd5bc0d1405aaecbb79ccd43d8ce'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:59:35.918067 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-698f134f-1ef4-4168-acd2-5bb6b3e58fa1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fad6fd5bc0d1405aaecbb79ccd43d8ce'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:59:35.918713 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-698f134f-1ef4-4168-acd2-5bb6b3e58fa1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fad6fd5bc0d1405aaecbb79ccd43d8ce'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:59:35.958871 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 841/1667] 10.222.0.64 () {58 vars in 1224 bytes} [Tue Aug 2 09:59:35 2022] DELETE /identity/v3/users/fad6fd5bc0d1405aaecbb79ccd43d8ce => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:59:35.960438 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63020667-c12a-4650-821d-6f450d35cba6 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:35.960787 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63020667-c12a-4650-821d-6f450d35cba6 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:35.961077 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63020667-c12a-4650-821d-6f450d35cba6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:35.966131 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3a0e6f-8e0b-4418-9034-d5c6097269d9 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:36.008453 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-63020667-c12a-4650-821d-6f450d35cba6 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:59:36.009710 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-63020667-c12a-4650-821d-6f450d35cba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:36.014292 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:59:36.032720 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:59:36.076615 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0b060b-28d8-4c6b-b6d4-e31c9b793408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:36.077254 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0b060b-28d8-4c6b-b6d4-e31c9b793408 admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:36.077545 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0b060b-28d8-4c6b-b6d4-e31c9b793408 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:36.077823 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0b060b-28d8-4c6b-b6d4-e31c9b793408 admin admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:36.078515 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0b060b-28d8-4c6b-b6d4-e31c9b793408 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:36.080756 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0b060b-28d8-4c6b-b6d4-e31c9b793408 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:36.124073 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 827/1668] 10.222.0.64 () {60 vars in 1144 bytes} [Tue Aug 2 09:59:35 2022] POST /identity/v3/users => generated 309 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 09:59:36.128125 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0b060b-28d8-4c6b-b6d4-e31c9b793408 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:36.130450 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-ef0b060b-28d8-4c6b-b6d4-e31c9b793408 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:36.132046 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 842/1669] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 09:59:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:59:36.148466 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0b060b-28d8-4c6b-b6d4-e31c9b793408 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:36.151465 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63020667-c12a-4650-821d-6f450d35cba6 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:36.154890 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-63020667-c12a-4650-821d-6f450d35cba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:36.156671 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d47727a-2ffc-4048-a339-b14c525b2f7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:36.160348 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d47727a-2ffc-4048-a339-b14c525b2f7f admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:36.160348 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d47727a-2ffc-4048-a339-b14c525b2f7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:36.160348 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d47727a-2ffc-4048-a339-b14c525b2f7f admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:36.160348 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d47727a-2ffc-4048-a339-b14c525b2f7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:36.162566 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d47727a-2ffc-4048-a339-b14c525b2f7f admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:36.194401 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 828/1670] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 09:59:36 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:59:36.202249 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d47727a-2ffc-4048-a339-b14c525b2f7f admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:36.242668 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d47727a-2ffc-4048-a339-b14c525b2f7f admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:36.256677 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96e6454-d7d1-441b-b3e1-ee08f22d0a59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:36.257373 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96e6454-d7d1-441b-b3e1-ee08f22d0a59 admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:36.257680 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96e6454-d7d1-441b-b3e1-ee08f22d0a59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:36.257943 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96e6454-d7d1-441b-b3e1-ee08f22d0a59 admin admin] PATH_INFO: `/v3/projects/851a765a9b094639914b087bd61e4db9/users/14a26f3d745742d9b08771b08412d3e4/roles/1000cf318fa14915b8091599eaead5de` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:36.283627 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63020667-c12a-4650-821d-6f450d35cba6 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:36.294280 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8212d5c6-ec1c-41cd-ad99-1eba6616981c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:36.294962 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8212d5c6-ec1c-41cd-ad99-1eba6616981c service neutron] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:36.295255 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8212d5c6-ec1c-41cd-ad99-1eba6616981c service neutron] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:36.295636 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8212d5c6-ec1c-41cd-ad99-1eba6616981c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:36.297865 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96e6454-d7d1-441b-b3e1-ee08f22d0a59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=851a765a9b094639914b087bd61e4db9, user_id=14a26f3d745742d9b08771b08412d3e4, role_id=1000cf318fa14915b8091599eaead5de)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:36.300594 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96e6454-d7d1-441b-b3e1-ee08f22d0a59 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:36.316784 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8212d5c6-ec1c-41cd-ad99-1eba6616981c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:36.318310 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8212d5c6-ec1c-41cd-ad99-1eba6616981c service neutron] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:36.334745 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 829/1671] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 09:59:36 2022] PUT /identity/v3/projects/851a765a9b094639914b087bd61e4db9/users/14a26f3d745742d9b08771b08412d3e4/roles/1000cf318fa14915b8091599eaead5de => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:59:36.336383 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96e6454-d7d1-441b-b3e1-ee08f22d0a59 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:36.340528 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-d96e6454-d7d1-441b-b3e1-ee08f22d0a59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:36.350560 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 843/1672] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 09:59:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 09:59:36.351552 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8212d5c6-ec1c-41cd-ad99-1eba6616981c service neutron] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:36.375473 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8212d5c6-ec1c-41cd-ad99-1eba6616981c service neutron] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:36.495977 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96e6454-d7d1-441b-b3e1-ee08f22d0a59 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:36.508032 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9184fce-f35c-4f13-a0f4-80b780ef40a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:36.508698 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9184fce-f35c-4f13-a0f4-80b780ef40a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:36.509589 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9184fce-f35c-4f13-a0f4-80b780ef40a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:36.511016 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9184fce-f35c-4f13-a0f4-80b780ef40a4 admin admin] PATH_INFO: `/v3/projects/844385826f9b4786b24f86575dbfb080` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:36.512943 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9184fce-f35c-4f13-a0f4-80b780ef40a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=844385826f9b4786b24f86575dbfb080)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:36.516412 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d63298-5f9c-476b-a0fb-43d0b2fd9044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:36.517502 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9184fce-f35c-4f13-a0f4-80b780ef40a4 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:36.518318 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d63298-5f9c-476b-a0fb-43d0b2fd9044 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:36.518714 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d63298-5f9c-476b-a0fb-43d0b2fd9044 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:36.520572 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d63298-5f9c-476b-a0fb-43d0b2fd9044 admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:36.521921 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d63298-5f9c-476b-a0fb-43d0b2fd9044 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:36.525161 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d63298-5f9c-476b-a0fb-43d0b2fd9044 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:36.550835 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 844/1673] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 09:59:36 2022] GET /identity/v3/roles => generated 1420 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:59:36.553588 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d63298-5f9c-476b-a0fb-43d0b2fd9044 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:36.579046 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d63298-5f9c-476b-a0fb-43d0b2fd9044 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:36.585010 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6722f6-57c0-494e-928c-428daeedfd5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:36.586252 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6722f6-57c0-494e-928c-428daeedfd5e admin admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:36.586844 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6722f6-57c0-494e-928c-428daeedfd5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:36.587273 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6722f6-57c0-494e-928c-428daeedfd5e admin admin] PATH_INFO: `/v3/projects/851a765a9b094639914b087bd61e4db9/users/14a26f3d745742d9b08771b08412d3e4/roles/1000cf318fa14915b8091599eaead5de` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:36.599991 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6722f6-57c0-494e-928c-428daeedfd5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=851a765a9b094639914b087bd61e4db9, user_id=14a26f3d745742d9b08771b08412d3e4, role_id=1000cf318fa14915b8091599eaead5de)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:36.608440 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6722f6-57c0-494e-928c-428daeedfd5e admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:36.625155 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-f9184fce-f35c-4f13-a0f4-80b780ef40a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '844385826f9b4786b24f86575dbfb080', 'request_id': 'req-f9184fce-f35c-4f13-a0f4-80b780ef40a4'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:59:36.639043 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-f9184fce-f35c-4f13-a0f4-80b780ef40a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '844385826f9b4786b24f86575dbfb080', 'request_id': 'req-f9184fce-f35c-4f13-a0f4-80b780ef40a4'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:59:36.639658 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 845/1674] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 09:59:36 2022] PUT /identity/v3/projects/851a765a9b094639914b087bd61e4db9/users/14a26f3d745742d9b08771b08412d3e4/roles/1000cf318fa14915b8091599eaead5de => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:59:36.646284 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-f9184fce-f35c-4f13-a0f4-80b780ef40a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '844385826f9b4786b24f86575dbfb080', 'request_id': 'req-f9184fce-f35c-4f13-a0f4-80b780ef40a4'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:59:36.649583 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d39f440-2b2e-41ac-878f-02e26d35dbe8 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:36.650032 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d39f440-2b2e-41ac-878f-02e26d35dbe8 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:36.650473 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d39f440-2b2e-41ac-878f-02e26d35dbe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:36.683139 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-f9184fce-f35c-4f13-a0f4-80b780ef40a4 admin admin] The token cache is being invalidate because project 844385826f9b4786b24f86575dbfb080 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80154) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:59:36.686741 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-f9184fce-f35c-4f13-a0f4-80b780ef40a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:59:36.690136 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-f9184fce-f35c-4f13-a0f4-80b780ef40a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:59:36.697081 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 830/1675] 10.222.0.64 () {58 vars in 1233 bytes} [Tue Aug 2 09:59:36 2022] DELETE /identity/v3/projects/844385826f9b4786b24f86575dbfb080 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:59:36.701661 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e041ddc-744b-4039-9cdc-a0cba6067dc2 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:36.702168 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e041ddc-744b-4039-9cdc-a0cba6067dc2 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:36.702596 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e041ddc-744b-4039-9cdc-a0cba6067dc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:36.726543 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-2d39f440-2b2e-41ac-878f-02e26d35dbe8 None None] MFA Rules not processed for user `14a26f3d745742d9b08771b08412d3e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:59:36.728958 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-2d39f440-2b2e-41ac-878f-02e26d35dbe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:36.735947 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-8e041ddc-744b-4039-9cdc-a0cba6067dc2 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:59:36.738202 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-8e041ddc-744b-4039-9cdc-a0cba6067dc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:36.758018 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:59:36.775524 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:59:36.873113 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 831/1676] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 09:59:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:59:36.876748 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ca8c2d-dbe7-4fed-af75-562d1aa10ed3 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:36.877176 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ca8c2d-dbe7-4fed-af75-562d1aa10ed3 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:36.877675 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ca8c2d-dbe7-4fed-af75-562d1aa10ed3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:36.884300 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 846/1677] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 09:59:36 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:59:36.896377 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d39f440-2b2e-41ac-878f-02e26d35dbe8 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:36.898477 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-2d39f440-2b2e-41ac-878f-02e26d35dbe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:36.942297 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-47ca8c2d-dbe7-4fed-af75-562d1aa10ed3 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:59:36.945223 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-47ca8c2d-dbe7-4fed-af75-562d1aa10ed3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:36.967650 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 832/1678] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 09:59:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:59:36.970800 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ca8c2d-dbe7-4fed-af75-562d1aa10ed3 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:36.991100 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ca8c2d-dbe7-4fed-af75-562d1aa10ed3 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:37.008292 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5960c6d2-b3db-4185-8ab8-b3279cfac76a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:37.009277 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5960c6d2-b3db-4185-8ab8-b3279cfac76a admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:37.009902 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5960c6d2-b3db-4185-8ab8-b3279cfac76a admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:37.013177 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5960c6d2-b3db-4185-8ab8-b3279cfac76a admin admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:37.015007 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5960c6d2-b3db-4185-8ab8-b3279cfac76a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:59:37.018750 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5960c6d2-b3db-4185-8ab8-b3279cfac76a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:37.021363 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5960c6d2-b3db-4185-8ab8-b3279cfac76a admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:37.031310 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d39f440-2b2e-41ac-878f-02e26d35dbe8 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:37.035539 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31b0d29-e378-4c5b-a55c-ee08af9bf436 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d7de62b9ee4039bdc79627ac7fb499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:37.036188 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31b0d29-e378-4c5b-a55c-ee08af9bf436 service cinder] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:37.036505 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31b0d29-e378-4c5b-a55c-ee08af9bf436 service cinder] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:37.036785 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31b0d29-e378-4c5b-a55c-ee08af9bf436 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:37.056458 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 833/1679] 10.222.0.64 () {58 vars in 1153 bytes} [Tue Aug 2 09:59:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:59:37.057122 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31b0d29-e378-4c5b-a55c-ee08af9bf436 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:37.058585 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31b0d29-e378-4c5b-a55c-ee08af9bf436 service cinder] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:37.060452 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5960c6d2-b3db-4185-8ab8-b3279cfac76a admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:37.081268 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 847/1680] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 09:59:36 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:59:37.085748 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5960c6d2-b3db-4185-8ab8-b3279cfac76a admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:37.091114 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f122c2-867a-478f-aa4d-fd7bebbf4aa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:37.094609 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f122c2-867a-478f-aa4d-fd7bebbf4aa1 admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:37.094735 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f122c2-867a-478f-aa4d-fd7bebbf4aa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:37.095245 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f122c2-867a-478f-aa4d-fd7bebbf4aa1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:37.096138 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f122c2-867a-478f-aa4d-fd7bebbf4aa1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:37.098749 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f122c2-867a-478f-aa4d-fd7bebbf4aa1 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:37.151162 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 834/1681] 10.222.0.64 () {60 vars in 1153 bytes} [Tue Aug 2 09:59:37 2022] POST /identity/v3/projects => generated 361 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:59:37.155844 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31b0d29-e378-4c5b-a55c-ee08af9bf436 service cinder] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:37.175735 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31b0d29-e378-4c5b-a55c-ee08af9bf436 service cinder] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:37.266545 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6000cd79-f9c5-49fc-a13e-90db7b4414bd None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:37.266848 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6000cd79-f9c5-49fc-a13e-90db7b4414bd None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:37.267704 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6000cd79-f9c5-49fc-a13e-90db7b4414bd None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:37.270330 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 835/1682] 10.222.0.64 () {54 vars in 912 bytes} [Tue Aug 2 09:59:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:59:37.277541 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e333db-68f7-4426-aec1-5a6fbddf857c None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:37.277838 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e333db-68f7-4426-aec1-5a6fbddf857c None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:37.278143 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e333db-68f7-4426-aec1-5a6fbddf857c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:37.351365 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da2b01e-cd8a-400f-9343-ba118d99a638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:37.352012 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da2b01e-cd8a-400f-9343-ba118d99a638 admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:37.352378 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da2b01e-cd8a-400f-9343-ba118d99a638 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:37.354809 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da2b01e-cd8a-400f-9343-ba118d99a638 admin admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:37.357669 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da2b01e-cd8a-400f-9343-ba118d99a638 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:37.364509 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da2b01e-cd8a-400f-9343-ba118d99a638 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:37.387538 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-23e333db-68f7-4426-aec1-5a6fbddf857c None None] MFA Rules not processed for user `ec31b188970947efb2ed1d4d9a31cadf`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:59:37.390106 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-23e333db-68f7-4426-aec1-5a6fbddf857c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:37.403201 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 848/1683] 10.222.0.64 () {60 vars in 1144 bytes} [Tue Aug 2 09:59:37 2022] POST /identity/v3/users => generated 281 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:59:37.406036 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da2b01e-cd8a-400f-9343-ba118d99a638 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:37.408840 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-5da2b01e-cd8a-400f-9343-ba118d99a638 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:37.431755 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da2b01e-cd8a-400f-9343-ba118d99a638 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:37.438323 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a59e59e-0b73-45ec-a534-1c0af6f99a2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:37.439278 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a59e59e-0b73-45ec-a534-1c0af6f99a2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:37.439723 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a59e59e-0b73-45ec-a534-1c0af6f99a2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:37.440168 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a59e59e-0b73-45ec-a534-1c0af6f99a2a admin admin] PATH_INFO: `/v3/users/b663b479be0b4ae1b9ba2b79bd6872cb` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:37.441906 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a59e59e-0b73-45ec-a534-1c0af6f99a2a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b663b479be0b4ae1b9ba2b79bd6872cb)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:37.444948 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a59e59e-0b73-45ec-a534-1c0af6f99a2a admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:37.531672 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-9a59e59e-0b73-45ec-a534-1c0af6f99a2a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b663b479be0b4ae1b9ba2b79bd6872cb'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:59:37.535201 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:59:37.540947 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 836/1684] 10.222.0.64 () {58 vars in 1009 bytes} [Tue Aug 2 09:59:37 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:59:37.541804 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e333db-68f7-4426-aec1-5a6fbddf857c None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:37.555021 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e333db-68f7-4426-aec1-5a6fbddf857c None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:37.558873 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-9a59e59e-0b73-45ec-a534-1c0af6f99a2a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b663b479be0b4ae1b9ba2b79bd6872cb'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:59:37.561303 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d520cf5-3ca9-4e12-9133-41e751b7b00d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:37.571290 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-9a59e59e-0b73-45ec-a534-1c0af6f99a2a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b663b479be0b4ae1b9ba2b79bd6872cb'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:59:37.572500 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-9a59e59e-0b73-45ec-a534-1c0af6f99a2a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b663b479be0b4ae1b9ba2b79bd6872cb'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:59:37.575066 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-9a59e59e-0b73-45ec-a534-1c0af6f99a2a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b663b479be0b4ae1b9ba2b79bd6872cb'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:59:37.575766 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d520cf5-3ca9-4e12-9133-41e751b7b00d admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:37.576885 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d520cf5-3ca9-4e12-9133-41e751b7b00d admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:37.577763 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d520cf5-3ca9-4e12-9133-41e751b7b00d admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:37.582938 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d520cf5-3ca9-4e12-9133-41e751b7b00d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:37.587471 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d520cf5-3ca9-4e12-9133-41e751b7b00d admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:37.611799 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 849/1686] 10.222.0.64 () {58 vars in 1224 bytes} [Tue Aug 2 09:59:37 2022] DELETE /identity/v3/users/b663b479be0b4ae1b9ba2b79bd6872cb => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:59:37.612448 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 837/1686] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 09:59:37 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:59:37.615401 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a59e59e-0b73-45ec-a534-1c0af6f99a2a admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:37.615926 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e325a0d-fabb-4118-bce4-5e77eb909ac9 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:37.616373 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e325a0d-fabb-4118-bce4-5e77eb909ac9 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:37.616779 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e325a0d-fabb-4118-bce4-5e77eb909ac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:37.618807 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-9a59e59e-0b73-45ec-a534-1c0af6f99a2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:37.672412 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-1e325a0d-fabb-4118-bce4-5e77eb909ac9 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:59:37.674092 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-1e325a0d-fabb-4118-bce4-5e77eb909ac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:37.774133 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a59e59e-0b73-45ec-a534-1c0af6f99a2a admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:37.780273 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7105152-e127-4dbf-a724-08df9d5be320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:37.781055 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7105152-e127-4dbf-a724-08df9d5be320 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:37.781291 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7105152-e127-4dbf-a724-08df9d5be320 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:37.781576 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7105152-e127-4dbf-a724-08df9d5be320 admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:37.783144 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7105152-e127-4dbf-a724-08df9d5be320 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:37.785073 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7105152-e127-4dbf-a724-08df9d5be320 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:37.803220 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 850/1687] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 09:59:37 2022] GET /identity/v3/roles => generated 1420 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:59:37.803422 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7105152-e127-4dbf-a724-08df9d5be320 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:37.806038 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-f7105152-e127-4dbf-a724-08df9d5be320 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:37.839383 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 838/1688] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 09:59:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:59:37.843257 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e325a0d-fabb-4118-bce4-5e77eb909ac9 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:37.856426 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:59:37.872315 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e325a0d-fabb-4118-bce4-5e77eb909ac9 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:37.888313 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040c61aa-2837-4aaf-98f0-e4f8be695e23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:37.888313 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040c61aa-2837-4aaf-98f0-e4f8be695e23 admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:37.888313 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040c61aa-2837-4aaf-98f0-e4f8be695e23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:37.888313 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040c61aa-2837-4aaf-98f0-e4f8be695e23 admin admin] PATH_INFO: `/v3/projects/e2220775ea3341578779d01c08a53b6a/users/51c282df03f645568e91eae73bb24995/roles/66c1074780954f8e87ed11a5c4e8c36a` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:37.917877 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7105152-e127-4dbf-a724-08df9d5be320 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:37.928521 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040c61aa-2837-4aaf-98f0-e4f8be695e23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2220775ea3341578779d01c08a53b6a, user_id=51c282df03f645568e91eae73bb24995, role_id=66c1074780954f8e87ed11a5c4e8c36a)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:37.930910 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bdb961-3b60-4ced-bdd9-7b69527d7e80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:37.932594 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bdb961-3b60-4ced-bdd9-7b69527d7e80 service neutron] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:37.932856 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040c61aa-2837-4aaf-98f0-e4f8be695e23 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:37.934514 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bdb961-3b60-4ced-bdd9-7b69527d7e80 service neutron] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:37.934828 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bdb961-3b60-4ced-bdd9-7b69527d7e80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:37.941580 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-16bdb961-3b60-4ced-bdd9-7b69527d7e80 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:37.959266 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 839/1689] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 09:59:37 2022] PUT /identity/v3/projects/e2220775ea3341578779d01c08a53b6a/users/51c282df03f645568e91eae73bb24995/roles/66c1074780954f8e87ed11a5c4e8c36a => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:59:37.961196 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040c61aa-2837-4aaf-98f0-e4f8be695e23 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:37.969299 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bdb961-3b60-4ced-bdd9-7b69527d7e80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:37.972459 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bdb961-3b60-4ced-bdd9-7b69527d7e80 service neutron] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:37.996483 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 851/1690] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 09:59:37 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:59:37.998058 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba3e8fc-1027-4be9-8f4a-64bfe670a0d7 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:37.998850 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba3e8fc-1027-4be9-8f4a-64bfe670a0d7 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:37.999134 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba3e8fc-1027-4be9-8f4a-64bfe670a0d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:38.038325 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040c61aa-2837-4aaf-98f0-e4f8be695e23 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:38.042909 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea61b9d-0813-429c-8eee-266bbaacccdf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:38.044071 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea61b9d-0813-429c-8eee-266bbaacccdf service neutron] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:38.044573 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea61b9d-0813-429c-8eee-266bbaacccdf service neutron] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:38.045566 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea61b9d-0813-429c-8eee-266bbaacccdf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:38.092727 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-dba3e8fc-1027-4be9-8f4a-64bfe670a0d7 None None] MFA Rules not processed for user `51c282df03f645568e91eae73bb24995`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:59:38.094275 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-dba3e8fc-1027-4be9-8f4a-64bfe670a0d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:38.162037 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea61b9d-0813-429c-8eee-266bbaacccdf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:38.164175 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea61b9d-0813-429c-8eee-266bbaacccdf service neutron] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:38.187249 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 840/1691] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 09:59:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:59:38.217740 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:59:38.231909 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 852/1692] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 09:59:37 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:59:38.238178 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea61b9d-0813-429c-8eee-266bbaacccdf service neutron] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:38.250782 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea61b9d-0813-429c-8eee-266bbaacccdf service neutron] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:38.256027 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a170a0-9652-4c43-a22a-6fb6dc67ef15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:38.256706 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a170a0-9652-4c43-a22a-6fb6dc67ef15 admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:38.256998 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a170a0-9652-4c43-a22a-6fb6dc67ef15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:38.257282 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a170a0-9652-4c43-a22a-6fb6dc67ef15 admin admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:38.258125 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a170a0-9652-4c43-a22a-6fb6dc67ef15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:38.260063 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a170a0-9652-4c43-a22a-6fb6dc67ef15 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:38.297304 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 841/1693] 10.222.0.64 () {60 vars in 1153 bytes} [Tue Aug 2 09:59:38 2022] POST /identity/v3/projects => generated 365 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:59:38.302293 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba3e8fc-1027-4be9-8f4a-64bfe670a0d7 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:38.318328 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba3e8fc-1027-4be9-8f4a-64bfe670a0d7 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:38.487118 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a170a0-9652-4c43-a22a-6fb6dc67ef15 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:38.489590 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-66a170a0-9652-4c43-a22a-6fb6dc67ef15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:38.505300 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d52744-1ea6-4fb3-b705-4b05530f909f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:38.506091 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d52744-1ea6-4fb3-b705-4b05530f909f admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:38.506270 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d52744-1ea6-4fb3-b705-4b05530f909f admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:38.508546 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d52744-1ea6-4fb3-b705-4b05530f909f admin admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:38.509271 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d52744-1ea6-4fb3-b705-4b05530f909f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:38.509744 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a170a0-9652-4c43-a22a-6fb6dc67ef15 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:38.511252 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d52744-1ea6-4fb3-b705-4b05530f909f admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:38.517697 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356a47ae-9ae9-437d-8294-7c5c48d99712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:38.518831 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356a47ae-9ae9-437d-8294-7c5c48d99712 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:38.519614 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356a47ae-9ae9-437d-8294-7c5c48d99712 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:38.520473 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356a47ae-9ae9-437d-8294-7c5c48d99712 admin admin] PATH_INFO: `/v3/projects/dfc5594b6a9b4a118e1450ee0834003a` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:38.522648 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356a47ae-9ae9-437d-8294-7c5c48d99712 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dfc5594b6a9b4a118e1450ee0834003a)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:38.525758 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356a47ae-9ae9-437d-8294-7c5c48d99712 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:38.572429 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 853/1694] 10.222.0.64 () {60 vars in 1144 bytes} [Tue Aug 2 09:59:38 2022] POST /identity/v3/users => generated 289 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:59:38.578177 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d52744-1ea6-4fb3-b705-4b05530f909f admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:38.601007 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d52744-1ea6-4fb3-b705-4b05530f909f admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:38.613996 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b0460d-4d0d-4514-b7e6-d624835bb263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:38.614635 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b0460d-4d0d-4514-b7e6-d624835bb263 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:38.614928 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b0460d-4d0d-4514-b7e6-d624835bb263 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:38.615211 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b0460d-4d0d-4514-b7e6-d624835bb263 admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:38.615921 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b0460d-4d0d-4514-b7e6-d624835bb263 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:38.617755 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b0460d-4d0d-4514-b7e6-d624835bb263 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:38.649184 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 854/1695] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 09:59:38 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 09:59:38.672580 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-356a47ae-9ae9-437d-8294-7c5c48d99712 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dfc5594b6a9b4a118e1450ee0834003a', 'request_id': 'req-356a47ae-9ae9-437d-8294-7c5c48d99712'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:59:38.677160 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b0460d-4d0d-4514-b7e6-d624835bb263 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:38.684476 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-356a47ae-9ae9-437d-8294-7c5c48d99712 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dfc5594b6a9b4a118e1450ee0834003a', 'request_id': 'req-356a47ae-9ae9-437d-8294-7c5c48d99712'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:59:38.689782 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:59:38.702571 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-356a47ae-9ae9-437d-8294-7c5c48d99712 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dfc5594b6a9b4a118e1450ee0834003a', 'request_id': 'req-356a47ae-9ae9-437d-8294-7c5c48d99712'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:59:38.713911 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b0460d-4d0d-4514-b7e6-d624835bb263 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:38.742337 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-356a47ae-9ae9-437d-8294-7c5c48d99712 admin admin] The token cache is being invalidate because project dfc5594b6a9b4a118e1450ee0834003a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80154) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:59:38.742787 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-356a47ae-9ae9-437d-8294-7c5c48d99712 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:59:38.745111 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-356a47ae-9ae9-437d-8294-7c5c48d99712 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:59:38.747719 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 842/1696] 10.222.0.64 () {58 vars in 1233 bytes} [Tue Aug 2 09:59:38 2022] DELETE /identity/v3/projects/dfc5594b6a9b4a118e1450ee0834003a => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:59:38.752922 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356a47ae-9ae9-437d-8294-7c5c48d99712 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:38.756282 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-356a47ae-9ae9-437d-8294-7c5c48d99712 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:38.928219 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4181ef-504c-4627-ba5f-b123035d6411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:38.929545 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4181ef-504c-4627-ba5f-b123035d6411 admin admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:38.929980 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4181ef-504c-4627-ba5f-b123035d6411 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:38.930950 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4181ef-504c-4627-ba5f-b123035d6411 admin admin] PATH_INFO: `/v3/projects/b87c162bd2a9412486babc73fa284607/users/97d3049e6a994e9aaa8d1cd8ecbdeea5/roles/1000cf318fa14915b8091599eaead5de` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:38.932549 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356a47ae-9ae9-437d-8294-7c5c48d99712 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:38.948279 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb562b1-1fe3-4bd3-9de0-2d6e7616ec36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:38.949062 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb562b1-1fe3-4bd3-9de0-2d6e7616ec36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:38.949456 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb562b1-1fe3-4bd3-9de0-2d6e7616ec36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:38.949839 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb562b1-1fe3-4bd3-9de0-2d6e7616ec36 admin admin] PATH_INFO: `/v3/users/e1c4cd72c15d48139498b3dfa8b74241` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:38.952801 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb562b1-1fe3-4bd3-9de0-2d6e7616ec36 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1c4cd72c15d48139498b3dfa8b74241)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:38.955540 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb562b1-1fe3-4bd3-9de0-2d6e7616ec36 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:38.977117 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4181ef-504c-4627-ba5f-b123035d6411 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b87c162bd2a9412486babc73fa284607, user_id=97d3049e6a994e9aaa8d1cd8ecbdeea5, role_id=1000cf318fa14915b8091599eaead5de)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:38.980069 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4181ef-504c-4627-ba5f-b123035d6411 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:39.024236 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 855/1697] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 09:59:38 2022] PUT /identity/v3/projects/b87c162bd2a9412486babc73fa284607/users/97d3049e6a994e9aaa8d1cd8ecbdeea5/roles/1000cf318fa14915b8091599eaead5de => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:59:39.027755 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4181ef-504c-4627-ba5f-b123035d6411 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:39.030747 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-dd4181ef-504c-4627-ba5f-b123035d6411 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:39.067636 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-8bb562b1-1fe3-4bd3-9de0-2d6e7616ec36 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1c4cd72c15d48139498b3dfa8b74241'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:59:39.085088 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-8bb562b1-1fe3-4bd3-9de0-2d6e7616ec36 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1c4cd72c15d48139498b3dfa8b74241'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:59:39.086203 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-8bb562b1-1fe3-4bd3-9de0-2d6e7616ec36 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1c4cd72c15d48139498b3dfa8b74241'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:59:39.108961 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-8bb562b1-1fe3-4bd3-9de0-2d6e7616ec36 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1c4cd72c15d48139498b3dfa8b74241'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:59:39.130739 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-8bb562b1-1fe3-4bd3-9de0-2d6e7616ec36 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1c4cd72c15d48139498b3dfa8b74241'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:59:39.144918 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 843/1698] 10.222.0.64 () {58 vars in 1224 bytes} [Tue Aug 2 09:59:38 2022] DELETE /identity/v3/users/e1c4cd72c15d48139498b3dfa8b74241 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:59:39.179875 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:59:39.192227 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4181ef-504c-4627-ba5f-b123035d6411 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:39.312084 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6028c84c-82bc-4e63-9380-ebc390f8939a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:39.313364 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6028c84c-82bc-4e63-9380-ebc390f8939a admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:39.313364 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6028c84c-82bc-4e63-9380-ebc390f8939a admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:39.313364 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6028c84c-82bc-4e63-9380-ebc390f8939a admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:39.314277 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6028c84c-82bc-4e63-9380-ebc390f8939a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:39.316083 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6028c84c-82bc-4e63-9380-ebc390f8939a admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:39.329887 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 856/1699] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 09:59:39 2022] GET /identity/v3/roles => generated 1420 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:59:39.333697 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb562b1-1fe3-4bd3-9de0-2d6e7616ec36 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:39.335258 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-8bb562b1-1fe3-4bd3-9de0-2d6e7616ec36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:39.350403 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb562b1-1fe3-4bd3-9de0-2d6e7616ec36 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:39.356994 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2b864a-383c-4675-a22e-9997d73e783a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:39.357812 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2b864a-383c-4675-a22e-9997d73e783a admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:39.358181 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2b864a-383c-4675-a22e-9997d73e783a admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:39.358532 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2b864a-383c-4675-a22e-9997d73e783a admin admin] PATH_INFO: `/v3/projects/b87c162bd2a9412486babc73fa284607/users/97d3049e6a994e9aaa8d1cd8ecbdeea5/roles/1000cf318fa14915b8091599eaead5de` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:39.360796 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2b864a-383c-4675-a22e-9997d73e783a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b87c162bd2a9412486babc73fa284607, user_id=97d3049e6a994e9aaa8d1cd8ecbdeea5, role_id=1000cf318fa14915b8091599eaead5de)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:39.364067 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2b864a-383c-4675-a22e-9997d73e783a admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:39.383604 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 844/1700] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 09:59:39 2022] PUT /identity/v3/projects/b87c162bd2a9412486babc73fa284607/users/97d3049e6a994e9aaa8d1cd8ecbdeea5/roles/1000cf318fa14915b8091599eaead5de => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:59:39.394781 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3a0600-e66a-4f80-b17e-fae92e9f5960 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:39.395063 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3a0600-e66a-4f80-b17e-fae92e9f5960 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:39.395478 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3a0600-e66a-4f80-b17e-fae92e9f5960 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:39.455168 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-1e3a0600-e66a-4f80-b17e-fae92e9f5960 None None] MFA Rules not processed for user `97d3049e6a994e9aaa8d1cd8ecbdeea5`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:59:39.456815 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-1e3a0600-e66a-4f80-b17e-fae92e9f5960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:39.676069 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 857/1701] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 09:59:39 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:59:39.687651 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2b864a-383c-4675-a22e-9997d73e783a admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:39.689582 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-8b2b864a-383c-4675-a22e-9997d73e783a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:39.868377 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2b864a-383c-4675-a22e-9997d73e783a admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:39.874283 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4159c347-f170-40be-9837-f1ec7216870b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d7de62b9ee4039bdc79627ac7fb499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:39.876280 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4159c347-f170-40be-9837-f1ec7216870b service cinder] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:39.876930 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4159c347-f170-40be-9837-f1ec7216870b service cinder] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:39.878034 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4159c347-f170-40be-9837-f1ec7216870b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:39.907274 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4159c347-f170-40be-9837-f1ec7216870b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:39.908940 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4159c347-f170-40be-9837-f1ec7216870b service cinder] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:39.925670 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=86,keystone:INSERT=3,keystone:DELETE=11 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:59:39.940815 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 845/1702] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 09:59:39 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:59:40.090908 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1783b71-ab21-4cbd-ae01-df9262e0f85a None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:40.092955 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1783b71-ab21-4cbd-ae01-df9262e0f85a None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:40.093444 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1783b71-ab21-4cbd-ae01-df9262e0f85a None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:40.095867 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 858/1703] 10.222.0.64 () {54 vars in 961 bytes} [Tue Aug 2 09:59:40 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:59:40.106420 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2df366-6a0e-4fd0-aeec-ad593d95b156 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:40.106722 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2df366-6a0e-4fd0-aeec-ad593d95b156 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:40.107010 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2df366-6a0e-4fd0-aeec-ad593d95b156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:40.108270 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1783b71-ab21-4cbd-ae01-df9262e0f85a None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:40.126430 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1783b71-ab21-4cbd-ae01-df9262e0f85a None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:40.137639 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f7a2c7-3ee6-46c9-8b1d-e23858c4cd53 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d7de62b9ee4039bdc79627ac7fb499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:40.138603 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f7a2c7-3ee6-46c9-8b1d-e23858c4cd53 service cinder] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:40.139151 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f7a2c7-3ee6-46c9-8b1d-e23858c4cd53 service cinder] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:40.139770 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f7a2c7-3ee6-46c9-8b1d-e23858c4cd53 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:40.142548 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-73f7a2c7-3ee6-46c9-8b1d-e23858c4cd53 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:40.178696 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-1b2df366-6a0e-4fd0-aeec-ad593d95b156 None None] MFA Rules not processed for user `b770f073c86b4d88837a8b1f796a8b1d`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:59:40.183347 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-1b2df366-6a0e-4fd0-aeec-ad593d95b156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:40.338054 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 846/1704] 10.222.0.64 () {58 vars in 1058 bytes} [Tue Aug 2 09:59:40 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Aug 02 09:59:40.340419 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efc8576-90c9-43a8-8c1b-1c446ff2d096 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:40.340961 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efc8576-90c9-43a8-8c1b-1c446ff2d096 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:40.341091 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efc8576-90c9-43a8-8c1b-1c446ff2d096 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:40.347916 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f7a2c7-3ee6-46c9-8b1d-e23858c4cd53 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:40.349741 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f7a2c7-3ee6-46c9-8b1d-e23858c4cd53 service cinder] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:40.372119 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-6efc8576-90c9-43a8-8c1b-1c446ff2d096 None None] MFA Rules not processed for user `5bbba5ae48cc4ec39467df736ad1bbad`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:59:40.375339 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-6efc8576-90c9-43a8-8c1b-1c446ff2d096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:40.379710 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 859/1705] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 09:59:40 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 09:59:40.381525 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f7a2c7-3ee6-46c9-8b1d-e23858c4cd53 service cinder] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:40.429521 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f7a2c7-3ee6-46c9-8b1d-e23858c4cd53 service cinder] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:40.446039 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1869a761-6b15-4810-b253-bb9a505ffce2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b770f073c86b4d88837a8b1f796a8b1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:40.446685 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1869a761-6b15-4810-b253-bb9a505ffce2 service placement] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:40.446974 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1869a761-6b15-4810-b253-bb9a505ffce2 service placement] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:40.447326 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1869a761-6b15-4810-b253-bb9a505ffce2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:40.457401 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-1869a761-6b15-4810-b253-bb9a505ffce2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:40.496283 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1869a761-6b15-4810-b253-bb9a505ffce2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:40.497900 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1869a761-6b15-4810-b253-bb9a505ffce2 service placement] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:40.531503 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 847/1706] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 09:59:40 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:59:40.539551 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 860/1707] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 09:59:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:59:42.100927 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efc8576-90c9-43a8-8c1b-1c446ff2d096 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:42.104301 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-6efc8576-90c9-43a8-8c1b-1c446ff2d096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:42.395646 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c81bd2-8701-4451-bafb-198bc5cde3d4 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:42.396933 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c81bd2-8701-4451-bafb-198bc5cde3d4 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:42.397521 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c81bd2-8701-4451-bafb-198bc5cde3d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:42.420922 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efc8576-90c9-43a8-8c1b-1c446ff2d096 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:42.433254 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c8c58d-31ce-45f3-a91b-664eff915997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:42.434538 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c8c58d-31ce-45f3-a91b-664eff915997 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:42.436016 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c8c58d-31ce-45f3-a91b-664eff915997 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:42.436773 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c8c58d-31ce-45f3-a91b-664eff915997 admin admin] PATH_INFO: `/v3/projects/6b0cf8a7576845d39063552bf712ef25` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:42.441256 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:59:42.442690 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c8c58d-31ce-45f3-a91b-664eff915997 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b0cf8a7576845d39063552bf712ef25)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:42.448400 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c8c58d-31ce-45f3-a91b-664eff915997 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:42.467616 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-37c81bd2-8701-4451-bafb-198bc5cde3d4 None None] MFA Rules not processed for user `c94f607a834f43e7a0073fe27d265901`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:59:42.472206 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-37c81bd2-8701-4451-bafb-198bc5cde3d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:42.571250 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:59:42.601060 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-41c8c58d-31ce-45f3-a91b-664eff915997 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b0cf8a7576845d39063552bf712ef25', 'request_id': 'req-41c8c58d-31ce-45f3-a91b-664eff915997'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:59:42.637203 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-41c8c58d-31ce-45f3-a91b-664eff915997 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b0cf8a7576845d39063552bf712ef25', 'request_id': 'req-41c8c58d-31ce-45f3-a91b-664eff915997'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:59:42.662272 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-41c8c58d-31ce-45f3-a91b-664eff915997 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b0cf8a7576845d39063552bf712ef25', 'request_id': 'req-41c8c58d-31ce-45f3-a91b-664eff915997'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:59:42.760917 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-41c8c58d-31ce-45f3-a91b-664eff915997 admin admin] The token cache is being invalidate because project 6b0cf8a7576845d39063552bf712ef25 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80154) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:59:42.764569 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-41c8c58d-31ce-45f3-a91b-664eff915997 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:59:42.767295 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-41c8c58d-31ce-45f3-a91b-664eff915997 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:59:42.775603 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 848/1708] 10.222.0.64 () {58 vars in 1233 bytes} [Tue Aug 2 09:59:42 2022] DELETE /identity/v3/projects/6b0cf8a7576845d39063552bf712ef25 => generated 0 bytes in 679 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:59:42.781498 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1ba3a3-0976-46ed-b845-7a2bcdd03516 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:42.783353 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1ba3a3-0976-46ed-b845-7a2bcdd03516 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:42.792587 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 861/1709] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 09:59:42 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 398 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:59:42.793397 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1ba3a3-0976-46ed-b845-7a2bcdd03516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:42.809955 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c81bd2-8701-4451-bafb-198bc5cde3d4 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:42.814399 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-37c81bd2-8701-4451-bafb-198bc5cde3d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:42.882280 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-3e1ba3a3-0976-46ed-b845-7a2bcdd03516 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:59:42.893349 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-3e1ba3a3-0976-46ed-b845-7a2bcdd03516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:43.087794 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 849/1710] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 09:59:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:59:43.089419 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e388b890-01c9-49d0-882a-07279b58ad5a None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:43.089714 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e388b890-01c9-49d0-882a-07279b58ad5a None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:43.089993 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e388b890-01c9-49d0-882a-07279b58ad5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:43.127553 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-e388b890-01c9-49d0-882a-07279b58ad5a None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:59:43.129509 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-e388b890-01c9-49d0-882a-07279b58ad5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:43.141025 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 850/1711] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 09:59:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:59:43.146562 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e388b890-01c9-49d0-882a-07279b58ad5a None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:43.160486 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e388b890-01c9-49d0-882a-07279b58ad5a None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:43.183647 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c81bd2-8701-4451-bafb-198bc5cde3d4 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:43.187524 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddc498d-ddf2-4b0b-af47-471b686fa541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:43.188158 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddc498d-ddf2-4b0b-af47-471b686fa541 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:43.188497 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddc498d-ddf2-4b0b-af47-471b686fa541 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:43.188782 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddc498d-ddf2-4b0b-af47-471b686fa541 admin admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:43.189618 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddc498d-ddf2-4b0b-af47-471b686fa541 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:59:43.189942 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddc498d-ddf2-4b0b-af47-471b686fa541 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:43.191585 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddc498d-ddf2-4b0b-af47-471b686fa541 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:43.229408 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 851/1712] 10.222.0.64 () {58 vars in 1153 bytes} [Tue Aug 2 09:59:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:59:43.230548 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddc498d-ddf2-4b0b-af47-471b686fa541 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:43.235111 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-4ddc498d-ddf2-4b0b-af47-471b686fa541 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:43.271777 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddc498d-ddf2-4b0b-af47-471b686fa541 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:43.282453 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0b9746-8a8a-433d-ae38-d21a0e385fd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:43.284042 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0b9746-8a8a-433d-ae38-d21a0e385fd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:43.285746 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0b9746-8a8a-433d-ae38-d21a0e385fd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:43.286270 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0b9746-8a8a-433d-ae38-d21a0e385fd4 admin admin] PATH_INFO: `/v3/projects/463e77fb077b4c26a9f6ff5a19b4d7ee` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:43.292619 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0b9746-8a8a-433d-ae38-d21a0e385fd4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=463e77fb077b4c26a9f6ff5a19b4d7ee)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:43.299190 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0b9746-8a8a-433d-ae38-d21a0e385fd4 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:43.390100 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83da51d2-3398-4862-ae55-bd2a738b3c96 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99655e1f9ca24472a86cd26b0dbd1fc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:43.392987 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83da51d2-3398-4862-ae55-bd2a738b3c96 service nova] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:43.393450 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83da51d2-3398-4862-ae55-bd2a738b3c96 service nova] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:43.394054 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83da51d2-3398-4862-ae55-bd2a738b3c96 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:43.397577 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-83da51d2-3398-4862-ae55-bd2a738b3c96 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:43.409693 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-0a0b9746-8a8a-433d-ae38-d21a0e385fd4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '463e77fb077b4c26a9f6ff5a19b4d7ee', 'request_id': 'req-0a0b9746-8a8a-433d-ae38-d21a0e385fd4'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:59:43.426278 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-0a0b9746-8a8a-433d-ae38-d21a0e385fd4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '463e77fb077b4c26a9f6ff5a19b4d7ee', 'request_id': 'req-0a0b9746-8a8a-433d-ae38-d21a0e385fd4'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:59:43.436002 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-0a0b9746-8a8a-433d-ae38-d21a0e385fd4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '463e77fb077b4c26a9f6ff5a19b4d7ee', 'request_id': 'req-0a0b9746-8a8a-433d-ae38-d21a0e385fd4'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:59:43.463079 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:59:43.471465 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-0a0b9746-8a8a-433d-ae38-d21a0e385fd4 admin admin] The token cache is being invalidate because project 463e77fb077b4c26a9f6ff5a19b4d7ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80154) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:59:43.471928 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-0a0b9746-8a8a-433d-ae38-d21a0e385fd4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:59:43.472785 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-0a0b9746-8a8a-433d-ae38-d21a0e385fd4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:59:43.474477 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 852/1713] 10.222.0.64 () {58 vars in 1233 bytes} [Tue Aug 2 09:59:43 2022] DELETE /identity/v3/projects/463e77fb077b4c26a9f6ff5a19b4d7ee => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:59:43.479902 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0b9746-8a8a-433d-ae38-d21a0e385fd4 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:43.483740 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-0a0b9746-8a8a-433d-ae38-d21a0e385fd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:43.512429 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83da51d2-3398-4862-ae55-bd2a738b3c96 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:43.515625 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83da51d2-3398-4862-ae55-bd2a738b3c96 service nova] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:43.531961 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 862/1714] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 09:59:42 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 722 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:59:43.533329 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83da51d2-3398-4862-ae55-bd2a738b3c96 service nova] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:43.574058 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83da51d2-3398-4862-ae55-bd2a738b3c96 service nova] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:43.592707 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:59:43.660456 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0b9746-8a8a-433d-ae38-d21a0e385fd4 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:43.721735 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659301a5-f148-47c7-9c8d-ac60bb9b04fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:43.731315 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659301a5-f148-47c7-9c8d-ac60bb9b04fa admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:43.733150 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659301a5-f148-47c7-9c8d-ac60bb9b04fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:43.735476 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659301a5-f148-47c7-9c8d-ac60bb9b04fa admin admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:43.737772 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659301a5-f148-47c7-9c8d-ac60bb9b04fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:43.740339 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659301a5-f148-47c7-9c8d-ac60bb9b04fa admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:43.750980 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c1caea-d6ab-4554-9128-e665655d5f5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:43.751972 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c1caea-d6ab-4554-9128-e665655d5f5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:43.752404 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c1caea-d6ab-4554-9128-e665655d5f5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:43.752820 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c1caea-d6ab-4554-9128-e665655d5f5d admin admin] PATH_INFO: `/v3/users/14a26f3d745742d9b08771b08412d3e4` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:43.754733 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c1caea-d6ab-4554-9128-e665655d5f5d admin admin] RBAC: Authorizing `identity:delete_user(user_id=14a26f3d745742d9b08771b08412d3e4)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:43.758470 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c1caea-d6ab-4554-9128-e665655d5f5d admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:43.851843 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 853/1715] 10.222.0.64 () {60 vars in 1153 bytes} [Tue Aug 2 09:59:43 2022] POST /identity/v3/projects => generated 389 bytes in 374 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:59:43.853385 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659301a5-f148-47c7-9c8d-ac60bb9b04fa admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:43.874951 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-73c1caea-d6ab-4554-9128-e665655d5f5d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14a26f3d745742d9b08771b08412d3e4'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:59:43.903088 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-73c1caea-d6ab-4554-9128-e665655d5f5d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14a26f3d745742d9b08771b08412d3e4'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:59:43.924912 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-73c1caea-d6ab-4554-9128-e665655d5f5d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14a26f3d745742d9b08771b08412d3e4'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:59:43.925777 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-73c1caea-d6ab-4554-9128-e665655d5f5d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14a26f3d745742d9b08771b08412d3e4'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:59:43.926553 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-73c1caea-d6ab-4554-9128-e665655d5f5d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14a26f3d745742d9b08771b08412d3e4'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:59:43.960292 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 863/1716] 10.222.0.64 () {58 vars in 1224 bytes} [Tue Aug 2 09:59:43 2022] DELETE /identity/v3/users/14a26f3d745742d9b08771b08412d3e4 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:59:44.033253 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659301a5-f148-47c7-9c8d-ac60bb9b04fa admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:44.210413 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31bb901-5f14-40cc-8160-67416b604980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:44.211126 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31bb901-5f14-40cc-8160-67416b604980 admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:44.211341 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31bb901-5f14-40cc-8160-67416b604980 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:44.211620 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31bb901-5f14-40cc-8160-67416b604980 admin admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:44.212336 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31bb901-5f14-40cc-8160-67416b604980 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:44.222161 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31bb901-5f14-40cc-8160-67416b604980 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:44.308055 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 854/1717] 10.222.0.64 () {60 vars in 1144 bytes} [Tue Aug 2 09:59:43 2022] POST /identity/v3/users => generated 295 bytes in 455 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:59:44.313668 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c1caea-d6ab-4554-9128-e665655d5f5d admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:44.315080 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-73c1caea-d6ab-4554-9128-e665655d5f5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:44.341091 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c1caea-d6ab-4554-9128-e665655d5f5d admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:44.353451 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3c5b52-9706-4ec1-8042-834ab6a93e2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:44.355825 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3c5b52-9706-4ec1-8042-834ab6a93e2b admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:44.356208 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3c5b52-9706-4ec1-8042-834ab6a93e2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:44.356642 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3c5b52-9706-4ec1-8042-834ab6a93e2b admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:44.358460 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3c5b52-9706-4ec1-8042-834ab6a93e2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:44.364967 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3c5b52-9706-4ec1-8042-834ab6a93e2b admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:44.407740 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 864/1718] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 09:59:44 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:59:44.411574 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31bb901-5f14-40cc-8160-67416b604980 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:44.430751 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31bb901-5f14-40cc-8160-67416b604980 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:44.446697 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27af2318-2164-483a-92a3-d516fdae3d23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:44.448735 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27af2318-2164-483a-92a3-d516fdae3d23 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:44.449785 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27af2318-2164-483a-92a3-d516fdae3d23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:44.449785 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27af2318-2164-483a-92a3-d516fdae3d23 admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:44.453219 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27af2318-2164-483a-92a3-d516fdae3d23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:44.457684 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27af2318-2164-483a-92a3-d516fdae3d23 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:44.487713 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 855/1719] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 09:59:44 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:59:44.492031 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3c5b52-9706-4ec1-8042-834ab6a93e2b admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:44.530280 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3c5b52-9706-4ec1-8042-834ab6a93e2b admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:44.554372 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b5d0f5-4abf-44c9-85c1-e429d7359bd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:44.557770 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b5d0f5-4abf-44c9-85c1-e429d7359bd0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:44.561588 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b5d0f5-4abf-44c9-85c1-e429d7359bd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:44.562319 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b5d0f5-4abf-44c9-85c1-e429d7359bd0 admin admin] PATH_INFO: `/v3/projects/94f6fb15f2e947c08d3f4fa12c0978c3/users/f5f922138cb74e89af084186fc0af166/roles/66c1074780954f8e87ed11a5c4e8c36a` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:44.625215 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27af2318-2164-483a-92a3-d516fdae3d23 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:44.627225 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-27af2318-2164-483a-92a3-d516fdae3d23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:44.630924 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b5d0f5-4abf-44c9-85c1-e429d7359bd0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94f6fb15f2e947c08d3f4fa12c0978c3, user_id=f5f922138cb74e89af084186fc0af166, role_id=66c1074780954f8e87ed11a5c4e8c36a)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:44.636397 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b5d0f5-4abf-44c9-85c1-e429d7359bd0 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:44.651062 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27af2318-2164-483a-92a3-d516fdae3d23 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:44.663966 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56eb81bc-0dbe-4947-92ba-841a650c8ca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:44.665442 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eb81bc-0dbe-4947-92ba-841a650c8ca5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:44.666621 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 865/1720] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 09:59:44 2022] PUT /identity/v3/projects/94f6fb15f2e947c08d3f4fa12c0978c3/users/f5f922138cb74e89af084186fc0af166/roles/66c1074780954f8e87ed11a5c4e8c36a => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:59:44.667152 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eb81bc-0dbe-4947-92ba-841a650c8ca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:44.668190 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eb81bc-0dbe-4947-92ba-841a650c8ca5 admin admin] PATH_INFO: `/v3/users/51c282df03f645568e91eae73bb24995` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:44.669657 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56eb81bc-0dbe-4947-92ba-841a650c8ca5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=51c282df03f645568e91eae73bb24995)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:44.674204 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56eb81bc-0dbe-4947-92ba-841a650c8ca5 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:44.677415 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3475dcc1-2a0a-4aea-87c1-436c20a1adb0 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:44.677712 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3475dcc1-2a0a-4aea-87c1-436c20a1adb0 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:44.677987 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3475dcc1-2a0a-4aea-87c1-436c20a1adb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:44.754313 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-3475dcc1-2a0a-4aea-87c1-436c20a1adb0 None None] MFA Rules not processed for user `f5f922138cb74e89af084186fc0af166`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:59:44.757533 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-3475dcc1-2a0a-4aea-87c1-436c20a1adb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:44.773522 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:59:44.859695 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=91,keystone:INSERT=4,keystone:DELETE=5 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:59:44.895475 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-56eb81bc-0dbe-4947-92ba-841a650c8ca5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51c282df03f645568e91eae73bb24995'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:59:44.925501 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-56eb81bc-0dbe-4947-92ba-841a650c8ca5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51c282df03f645568e91eae73bb24995'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:59:44.927013 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-56eb81bc-0dbe-4947-92ba-841a650c8ca5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51c282df03f645568e91eae73bb24995'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:59:44.943881 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 866/1721] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 09:59:44 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:59:44.946537 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c876bc0f-39ed-4cc0-9ff8-3e80db1bbbf6 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:44.948355 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c876bc0f-39ed-4cc0-9ff8-3e80db1bbbf6 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:44.951294 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c876bc0f-39ed-4cc0-9ff8-3e80db1bbbf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:44.957710 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-56eb81bc-0dbe-4947-92ba-841a650c8ca5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51c282df03f645568e91eae73bb24995'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:59:44.994130 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-56eb81bc-0dbe-4947-92ba-841a650c8ca5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51c282df03f645568e91eae73bb24995'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:59:45.026641 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 856/1722] 10.222.0.64 () {58 vars in 1224 bytes} [Tue Aug 2 09:59:44 2022] DELETE /identity/v3/users/51c282df03f645568e91eae73bb24995 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:59:45.027659 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56eb81bc-0dbe-4947-92ba-841a650c8ca5 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:45.034459 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-56eb81bc-0dbe-4947-92ba-841a650c8ca5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:45.066112 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-c876bc0f-39ed-4cc0-9ff8-3e80db1bbbf6 None None] MFA Rules not processed for user `5338bc8c594e4bc9bc41019226bf8d87`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:59:45.068493 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-c876bc0f-39ed-4cc0-9ff8-3e80db1bbbf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:45.200799 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56eb81bc-0dbe-4947-92ba-841a650c8ca5 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:45.236541 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a18210-b35b-40f4-b134-4fafc15b88e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:45.243495 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a18210-b35b-40f4-b134-4fafc15b88e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:45.246071 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a18210-b35b-40f4-b134-4fafc15b88e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:45.246202 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a18210-b35b-40f4-b134-4fafc15b88e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:45.246926 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a18210-b35b-40f4-b134-4fafc15b88e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:45.251834 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a18210-b35b-40f4-b134-4fafc15b88e4 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:45.267034 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 867/1723] 10.222.0.64 () {58 vars in 1009 bytes} [Tue Aug 2 09:59:44 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 323 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:59:45.275915 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296ef467-dcad-4b4d-8806-7c1d39c8d700 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:45.276676 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296ef467-dcad-4b4d-8806-7c1d39c8d700 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:45.276970 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296ef467-dcad-4b4d-8806-7c1d39c8d700 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:45.353725 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-296ef467-dcad-4b4d-8806-7c1d39c8d700 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:59:45.356290 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 857/1724] 10.222.0.64 () {60 vars in 1153 bytes} [Tue Aug 2 09:59:45 2022] POST /identity/v3/projects => generated 387 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:59:45.358268 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-296ef467-dcad-4b4d-8806-7c1d39c8d700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:45.361094 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a18210-b35b-40f4-b134-4fafc15b88e4 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:45.537077 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 868/1725] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 09:59:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:59:45.537626 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a18210-b35b-40f4-b134-4fafc15b88e4 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:45.556650 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02151c26-f1bc-4afa-bcf4-7b8c4bdefef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:45.559205 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02151c26-f1bc-4afa-bcf4-7b8c4bdefef8 admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:45.559502 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02151c26-f1bc-4afa-bcf4-7b8c4bdefef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:45.565059 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02151c26-f1bc-4afa-bcf4-7b8c4bdefef8 admin admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:45.566416 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02151c26-f1bc-4afa-bcf4-7b8c4bdefef8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:45.581414 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02151c26-f1bc-4afa-bcf4-7b8c4bdefef8 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:45.606356 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296ef467-dcad-4b4d-8806-7c1d39c8d700 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:45.608110 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-296ef467-dcad-4b4d-8806-7c1d39c8d700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:45.682864 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 858/1726] 10.222.0.64 () {60 vars in 1144 bytes} [Tue Aug 2 09:59:45 2022] POST /identity/v3/users => generated 300 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:59:45.685932 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02151c26-f1bc-4afa-bcf4-7b8c4bdefef8 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:45.690928 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-02151c26-f1bc-4afa-bcf4-7b8c4bdefef8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:45.697086 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:59:45.876013 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:DELETE=5,keystone:SELECT=90,keystone:INSERT=5 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:59:45.986890 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02151c26-f1bc-4afa-bcf4-7b8c4bdefef8 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:46.003641 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661dd976-b1f4-45fd-9fae-17045a161531 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:46.005463 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661dd976-b1f4-45fd-9fae-17045a161531 service neutron] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:46.007097 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661dd976-b1f4-45fd-9fae-17045a161531 service neutron] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:46.008289 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661dd976-b1f4-45fd-9fae-17045a161531 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:46.036633 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296ef467-dcad-4b4d-8806-7c1d39c8d700 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:46.040388 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661dd976-b1f4-45fd-9fae-17045a161531 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:46.043414 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661dd976-b1f4-45fd-9fae-17045a161531 service neutron] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:46.073669 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e0ed4e-5f89-4b9d-964c-54a23ac9aff0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a371e3df3764a4cbb1b8368aa74bd65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:46.074337 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e0ed4e-5f89-4b9d-964c-54a23ac9aff0 service swift] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:46.074622 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e0ed4e-5f89-4b9d-964c-54a23ac9aff0 service swift] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:46.074896 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e0ed4e-5f89-4b9d-964c-54a23ac9aff0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:46.095097 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 859/1727] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 09:59:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:59:46.099323 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661dd976-b1f4-45fd-9fae-17045a161531 service neutron] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:46.135418 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661dd976-b1f4-45fd-9fae-17045a161531 service neutron] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:46.149802 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331b7517-cd32-4ff8-9d7c-ecbd63a6b3aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:46.150435 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331b7517-cd32-4ff8-9d7c-ecbd63a6b3aa admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:46.150771 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331b7517-cd32-4ff8-9d7c-ecbd63a6b3aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:46.151046 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331b7517-cd32-4ff8-9d7c-ecbd63a6b3aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:46.151750 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331b7517-cd32-4ff8-9d7c-ecbd63a6b3aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:46.153378 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331b7517-cd32-4ff8-9d7c-ecbd63a6b3aa admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:46.192670 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 860/1728] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 09:59:46 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:59:46.193564 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331b7517-cd32-4ff8-9d7c-ecbd63a6b3aa admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:46.206186 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-331b7517-cd32-4ff8-9d7c-ecbd63a6b3aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:46.244423 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331b7517-cd32-4ff8-9d7c-ecbd63a6b3aa admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:46.268144 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cf2794-1cb3-4feb-a76a-52afb7dd4b5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:46.268832 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cf2794-1cb3-4feb-a76a-52afb7dd4b5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:46.269115 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cf2794-1cb3-4feb-a76a-52afb7dd4b5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:46.269424 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cf2794-1cb3-4feb-a76a-52afb7dd4b5d admin admin] PATH_INFO: `/v3/projects/851a765a9b094639914b087bd61e4db9` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:46.273667 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cf2794-1cb3-4feb-a76a-52afb7dd4b5d admin admin] RBAC: Authorizing `identity:delete_project(project_id=851a765a9b094639914b087bd61e4db9)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:46.276422 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e0ed4e-5f89-4b9d-964c-54a23ac9aff0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:46.281081 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e0ed4e-5f89-4b9d-964c-54a23ac9aff0 service swift] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:46.285228 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cf2794-1cb3-4feb-a76a-52afb7dd4b5d admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:46.309244 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 869/1729] 10.222.0.64 () {60 vars in 1384 bytes} [Tue Aug 2 09:59:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 703 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 02 09:59:46.314085 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3fc41c-b016-4e4a-bf5c-10ac302ff4f8 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:46.315890 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3fc41c-b016-4e4a-bf5c-10ac302ff4f8 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:46.316822 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3fc41c-b016-4e4a-bf5c-10ac302ff4f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:46.462317 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-e5cf2794-1cb3-4feb-a76a-52afb7dd4b5d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '851a765a9b094639914b087bd61e4db9', 'request_id': 'req-e5cf2794-1cb3-4feb-a76a-52afb7dd4b5d'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:59:46.477304 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-ff3fc41c-b016-4e4a-bf5c-10ac302ff4f8 None None] MFA Rules not processed for user `982a01f0d3a5459a9add3f6f23db5db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:59:46.488509 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-ff3fc41c-b016-4e4a-bf5c-10ac302ff4f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:46.493668 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-e5cf2794-1cb3-4feb-a76a-52afb7dd4b5d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '851a765a9b094639914b087bd61e4db9', 'request_id': 'req-e5cf2794-1cb3-4feb-a76a-52afb7dd4b5d'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:59:46.537583 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-e5cf2794-1cb3-4feb-a76a-52afb7dd4b5d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '851a765a9b094639914b087bd61e4db9', 'request_id': 'req-e5cf2794-1cb3-4feb-a76a-52afb7dd4b5d'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:59:46.567874 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-e5cf2794-1cb3-4feb-a76a-52afb7dd4b5d admin admin] The token cache is being invalidate because project 851a765a9b094639914b087bd61e4db9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80154) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:59:46.568344 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-e5cf2794-1cb3-4feb-a76a-52afb7dd4b5d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:59:46.572117 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-e5cf2794-1cb3-4feb-a76a-52afb7dd4b5d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:59:46.578794 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 861/1730] 10.222.0.64 () {58 vars in 1233 bytes} [Tue Aug 2 09:59:46 2022] DELETE /identity/v3/projects/851a765a9b094639914b087bd61e4db9 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:59:46.579793 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cf2794-1cb3-4feb-a76a-52afb7dd4b5d admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:46.584328 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-e5cf2794-1cb3-4feb-a76a-52afb7dd4b5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:46.786624 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 870/1731] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 09:59:46 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 477 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:59:46.797615 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4757993-1846-4380-9958-4123731d5e96 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:46.797904 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4757993-1846-4380-9958-4123731d5e96 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:46.798938 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4757993-1846-4380-9958-4123731d5e96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:46.838765 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cf2794-1cb3-4feb-a76a-52afb7dd4b5d admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:46.863279 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63efc3d-5edf-461c-adcd-7d381a9f42bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:46.864184 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63efc3d-5edf-461c-adcd-7d381a9f42bb admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:46.869215 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63efc3d-5edf-461c-adcd-7d381a9f42bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:46.869215 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63efc3d-5edf-461c-adcd-7d381a9f42bb admin admin] PATH_INFO: `/v3/projects/7684bbebb0184c2f99fe4437a63207c3/users/9f843c14c3ef419ab4b05199bf131448/roles/1000cf318fa14915b8091599eaead5de` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:46.869545 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-c4757993-1846-4380-9958-4123731d5e96 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:59:46.870747 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-c4757993-1846-4380-9958-4123731d5e96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:46.910162 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 871/1732] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 09:59:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:59:46.910162 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4757993-1846-4380-9958-4123731d5e96 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:46.910841 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-c4757993-1846-4380-9958-4123731d5e96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:46.931392 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63efc3d-5edf-461c-adcd-7d381a9f42bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=7684bbebb0184c2f99fe4437a63207c3, user_id=9f843c14c3ef419ab4b05199bf131448, role_id=1000cf318fa14915b8091599eaead5de)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:46.947603 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63efc3d-5edf-461c-adcd-7d381a9f42bb admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:46.991704 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 862/1733] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 09:59:46 2022] PUT /identity/v3/projects/7684bbebb0184c2f99fe4437a63207c3/users/9f843c14c3ef419ab4b05199bf131448/roles/1000cf318fa14915b8091599eaead5de => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:59:46.993309 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6402ff-0ec7-4769-8cb4-6959085e2a77 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:46.995529 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6402ff-0ec7-4769-8cb4-6959085e2a77 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:46.996977 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6402ff-0ec7-4769-8cb4-6959085e2a77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:47.034100 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:59:47.092979 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-8d6402ff-0ec7-4769-8cb4-6959085e2a77 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:59:47.096059 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-8d6402ff-0ec7-4769-8cb4-6959085e2a77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:47.176703 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:59:47.205923 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4757993-1846-4380-9958-4123731d5e96 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:47.318108 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 863/1734] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 09:59:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:59:47.318997 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6402ff-0ec7-4769-8cb4-6959085e2a77 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:47.351795 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6402ff-0ec7-4769-8cb4-6959085e2a77 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:47.376489 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1090abc-135f-430b-b11a-773b802c122b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:47.377048 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1090abc-135f-430b-b11a-773b802c122b admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:47.377335 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1090abc-135f-430b-b11a-773b802c122b admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:47.377671 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1090abc-135f-430b-b11a-773b802c122b admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:47.378390 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1090abc-135f-430b-b11a-773b802c122b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:47.380134 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1090abc-135f-430b-b11a-773b802c122b admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:47.414074 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 864/1735] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 09:59:47 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:59:47.414074 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1090abc-135f-430b-b11a-773b802c122b admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:47.446889 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1090abc-135f-430b-b11a-773b802c122b admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:47.448442 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2a47a5-f1c5-482c-9de5-6f7dcb4b2254 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99655e1f9ca24472a86cd26b0dbd1fc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:47.450623 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2a47a5-f1c5-482c-9de5-6f7dcb4b2254 service nova] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:47.450623 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2a47a5-f1c5-482c-9de5-6f7dcb4b2254 service nova] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:47.450623 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2a47a5-f1c5-482c-9de5-6f7dcb4b2254 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:47.478112 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a4092f-f4c2-45c0-9cee-077b3393965b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:47.478666 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a4092f-f4c2-45c0-9cee-077b3393965b admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:47.478923 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a4092f-f4c2-45c0-9cee-077b3393965b admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:47.479277 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a4092f-f4c2-45c0-9cee-077b3393965b admin admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:47.480104 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a4092f-f4c2-45c0-9cee-077b3393965b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:59:47.480482 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a4092f-f4c2-45c0-9cee-077b3393965b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:47.482939 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a4092f-f4c2-45c0-9cee-077b3393965b admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:47.534621 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 865/1736] 10.222.0.64 () {58 vars in 1153 bytes} [Tue Aug 2 09:59:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:59:47.534859 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a4092f-f4c2-45c0-9cee-077b3393965b admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:47.576034 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a4092f-f4c2-45c0-9cee-077b3393965b admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:47.583016 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fa0e3a-fa9a-4e37-b597-13952aa507a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:47.583977 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fa0e3a-fa9a-4e37-b597-13952aa507a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:47.584408 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fa0e3a-fa9a-4e37-b597-13952aa507a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:47.584812 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fa0e3a-fa9a-4e37-b597-13952aa507a9 admin admin] PATH_INFO: `/v3/projects/7684bbebb0184c2f99fe4437a63207c3/users/9f843c14c3ef419ab4b05199bf131448/roles/1000cf318fa14915b8091599eaead5de` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:47.586977 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fa0e3a-fa9a-4e37-b597-13952aa507a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7684bbebb0184c2f99fe4437a63207c3, user_id=9f843c14c3ef419ab4b05199bf131448, role_id=1000cf318fa14915b8091599eaead5de)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:47.589960 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fa0e3a-fa9a-4e37-b597-13952aa507a9 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:47.618880 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 866/1737] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 09:59:47 2022] PUT /identity/v3/projects/7684bbebb0184c2f99fe4437a63207c3/users/9f843c14c3ef419ab4b05199bf131448/roles/1000cf318fa14915b8091599eaead5de => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:59:47.619879 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fa0e3a-fa9a-4e37-b597-13952aa507a9 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:47.645750 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2a47a5-f1c5-482c-9de5-6f7dcb4b2254 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:47.647413 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2a47a5-f1c5-482c-9de5-6f7dcb4b2254 service nova] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:47.684127 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 872/1738] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 09:59:46 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 778 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:59:47.687245 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c8269f-cceb-4ac4-8711-708b63baed6f None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:47.687538 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c8269f-cceb-4ac4-8711-708b63baed6f None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:47.687815 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c8269f-cceb-4ac4-8711-708b63baed6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:47.760699 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-b9c8269f-cceb-4ac4-8711-708b63baed6f None None] MFA Rules not processed for user `9f843c14c3ef419ab4b05199bf131448`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:59:47.762131 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-b9c8269f-cceb-4ac4-8711-708b63baed6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:47.766641 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fa0e3a-fa9a-4e37-b597-13952aa507a9 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:47.779789 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2764e61b-26d9-4aee-9ce9-55d8f0654215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:47.780449 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2764e61b-26d9-4aee-9ce9-55d8f0654215 admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:47.780735 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2764e61b-26d9-4aee-9ce9-55d8f0654215 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:47.781008 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2764e61b-26d9-4aee-9ce9-55d8f0654215 admin admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:47.781692 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2764e61b-26d9-4aee-9ce9-55d8f0654215 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:47.783616 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2764e61b-26d9-4aee-9ce9-55d8f0654215 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:47.889958 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:59:47.898457 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 867/1739] 10.222.0.64 () {60 vars in 1153 bytes} [Tue Aug 2 09:59:47 2022] POST /identity/v3/projects => generated 367 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:59:47.900961 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2764e61b-26d9-4aee-9ce9-55d8f0654215 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:47.940035 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 873/1740] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 09:59:47 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:59:47.952776 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c8269f-cceb-4ac4-8711-708b63baed6f None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:47.954770 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-b9c8269f-cceb-4ac4-8711-708b63baed6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:48.026305 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2764e61b-26d9-4aee-9ce9-55d8f0654215 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:48.034379 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdaa169-4753-45fa-b0b7-1c5bd8ebf3fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:48.062145 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdaa169-4753-45fa-b0b7-1c5bd8ebf3fa admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:48.084356 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdaa169-4753-45fa-b0b7-1c5bd8ebf3fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:48.084920 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdaa169-4753-45fa-b0b7-1c5bd8ebf3fa admin admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:48.086810 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdaa169-4753-45fa-b0b7-1c5bd8ebf3fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:48.088936 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdaa169-4753-45fa-b0b7-1c5bd8ebf3fa admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:48.115130 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c8269f-cceb-4ac4-8711-708b63baed6f None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:48.192428 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 868/1741] 10.222.0.64 () {60 vars in 1144 bytes} [Tue Aug 2 09:59:47 2022] POST /identity/v3/users => generated 284 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:59:48.198628 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdaa169-4753-45fa-b0b7-1c5bd8ebf3fa admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:48.210359 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-0cdaa169-4753-45fa-b0b7-1c5bd8ebf3fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:48.238259 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdaa169-4753-45fa-b0b7-1c5bd8ebf3fa admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:48.254872 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0619ccd-ba2a-4ac9-b990-48b5dc5d9997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:48.274655 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0619ccd-ba2a-4ac9-b990-48b5dc5d9997 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:48.276729 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0619ccd-ba2a-4ac9-b990-48b5dc5d9997 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:48.277313 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0619ccd-ba2a-4ac9-b990-48b5dc5d9997 admin admin] PATH_INFO: `/v3/projects/e2220775ea3341578779d01c08a53b6a` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:48.285535 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0619ccd-ba2a-4ac9-b990-48b5dc5d9997 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2220775ea3341578779d01c08a53b6a)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:48.291243 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0619ccd-ba2a-4ac9-b990-48b5dc5d9997 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:48.295724 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:59:48.378246 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df771044-b50a-489f-9ec7-38ce5e701fda service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d7de62b9ee4039bdc79627ac7fb499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:48.379012 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df771044-b50a-489f-9ec7-38ce5e701fda service cinder] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:48.379305 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df771044-b50a-489f-9ec7-38ce5e701fda service cinder] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:48.379574 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df771044-b50a-489f-9ec7-38ce5e701fda service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:48.453339 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-f0619ccd-ba2a-4ac9-b990-48b5dc5d9997 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2220775ea3341578779d01c08a53b6a', 'request_id': 'req-f0619ccd-ba2a-4ac9-b990-48b5dc5d9997'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:59:48.463993 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-f0619ccd-ba2a-4ac9-b990-48b5dc5d9997 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2220775ea3341578779d01c08a53b6a', 'request_id': 'req-f0619ccd-ba2a-4ac9-b990-48b5dc5d9997'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:59:48.472544 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-f0619ccd-ba2a-4ac9-b990-48b5dc5d9997 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2220775ea3341578779d01c08a53b6a', 'request_id': 'req-f0619ccd-ba2a-4ac9-b990-48b5dc5d9997'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:59:48.499902 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-f0619ccd-ba2a-4ac9-b990-48b5dc5d9997 admin admin] The token cache is being invalidate because project e2220775ea3341578779d01c08a53b6a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80154) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:59:48.500793 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-f0619ccd-ba2a-4ac9-b990-48b5dc5d9997 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:59:48.503767 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-f0619ccd-ba2a-4ac9-b990-48b5dc5d9997 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:59:48.514703 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 869/1742] 10.222.0.64 () {58 vars in 1233 bytes} [Tue Aug 2 09:59:48 2022] DELETE /identity/v3/projects/e2220775ea3341578779d01c08a53b6a => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:59:48.525531 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0619ccd-ba2a-4ac9-b990-48b5dc5d9997 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:48.534585 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-f0619ccd-ba2a-4ac9-b990-48b5dc5d9997 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:48.609284 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df771044-b50a-489f-9ec7-38ce5e701fda service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:48.610900 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df771044-b50a-489f-9ec7-38ce5e701fda service cinder] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:48.639469 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 874/1743] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 09:59:47 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 688 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 09:59:48.640626 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df771044-b50a-489f-9ec7-38ce5e701fda service cinder] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:48.675720 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df771044-b50a-489f-9ec7-38ce5e701fda service cinder] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:48.831339 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0619ccd-ba2a-4ac9-b990-48b5dc5d9997 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:48.852004 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08c6ab7-0869-429d-9d5b-2570eada5258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:48.853982 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08c6ab7-0869-429d-9d5b-2570eada5258 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:48.855791 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08c6ab7-0869-429d-9d5b-2570eada5258 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:48.857170 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08c6ab7-0869-429d-9d5b-2570eada5258 admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:48.861881 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08c6ab7-0869-429d-9d5b-2570eada5258 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:48.865133 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08c6ab7-0869-429d-9d5b-2570eada5258 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:48.897338 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:59:48.921571 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 870/1744] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 09:59:48 2022] GET /identity/v3/roles => generated 1420 bytes in 409 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:59:48.929392 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1453ef4-6f84-4b91-a64a-340a320f2742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:48.930106 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1453ef4-6f84-4b91-a64a-340a320f2742 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:48.930439 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1453ef4-6f84-4b91-a64a-340a320f2742 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:48.930721 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1453ef4-6f84-4b91-a64a-340a320f2742 admin admin] PATH_INFO: `/v3/users/97d3049e6a994e9aaa8d1cd8ecbdeea5` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:48.938837 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08c6ab7-0869-429d-9d5b-2570eada5258 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:48.939876 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1453ef4-6f84-4b91-a64a-340a320f2742 admin admin] RBAC: Authorizing `identity:delete_user(user_id=97d3049e6a994e9aaa8d1cd8ecbdeea5)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:48.954384 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1453ef4-6f84-4b91-a64a-340a320f2742 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:48.978031 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08c6ab7-0869-429d-9d5b-2570eada5258 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:48.993235 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294345a7-edbe-4897-93d8-c351d90f1ceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:48.993882 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294345a7-edbe-4897-93d8-c351d90f1ceb admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:48.994172 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294345a7-edbe-4897-93d8-c351d90f1ceb admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:48.994635 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294345a7-edbe-4897-93d8-c351d90f1ceb admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:48.995359 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294345a7-edbe-4897-93d8-c351d90f1ceb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:48.997334 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294345a7-edbe-4897-93d8-c351d90f1ceb admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:49.023128 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 871/1745] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 09:59:48 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:59:49.024471 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294345a7-edbe-4897-93d8-c351d90f1ceb admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:49.033079 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-294345a7-edbe-4897-93d8-c351d90f1ceb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:49.056520 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-d1453ef4-6f84-4b91-a64a-340a320f2742 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97d3049e6a994e9aaa8d1cd8ecbdeea5'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:59:49.071702 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294345a7-edbe-4897-93d8-c351d90f1ceb admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:49.072643 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-d1453ef4-6f84-4b91-a64a-340a320f2742 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97d3049e6a994e9aaa8d1cd8ecbdeea5'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:59:49.090606 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-d1453ef4-6f84-4b91-a64a-340a320f2742 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97d3049e6a994e9aaa8d1cd8ecbdeea5'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:59:49.092029 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-d1453ef4-6f84-4b91-a64a-340a320f2742 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97d3049e6a994e9aaa8d1cd8ecbdeea5'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:59:49.092706 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-d1453ef4-6f84-4b91-a64a-340a320f2742 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97d3049e6a994e9aaa8d1cd8ecbdeea5'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:59:49.094791 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1f6bb2-b8c5-4409-870c-f02c5cd1fa0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:49.095715 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1f6bb2-b8c5-4409-870c-f02c5cd1fa0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:49.096165 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1f6bb2-b8c5-4409-870c-f02c5cd1fa0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:49.099059 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1f6bb2-b8c5-4409-870c-f02c5cd1fa0e admin admin] PATH_INFO: `/v3/users/f5f922138cb74e89af084186fc0af166` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:49.103175 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1f6bb2-b8c5-4409-870c-f02c5cd1fa0e admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5f922138cb74e89af084186fc0af166)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:49.118676 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1f6bb2-b8c5-4409-870c-f02c5cd1fa0e admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:49.159385 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 875/1746] 10.222.0.64 () {58 vars in 1224 bytes} [Tue Aug 2 09:59:48 2022] DELETE /identity/v3/users/97d3049e6a994e9aaa8d1cd8ecbdeea5 => generated 0 bytes in 516 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:59:49.159385 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1453ef4-6f84-4b91-a64a-340a320f2742 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:49.172476 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-d1453ef4-6f84-4b91-a64a-340a320f2742 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:49.286218 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-4d1f6bb2-b8c5-4409-870c-f02c5cd1fa0e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5f922138cb74e89af084186fc0af166'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:59:49.305177 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-4d1f6bb2-b8c5-4409-870c-f02c5cd1fa0e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5f922138cb74e89af084186fc0af166'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:59:49.306289 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-4d1f6bb2-b8c5-4409-870c-f02c5cd1fa0e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5f922138cb74e89af084186fc0af166'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:59:49.332580 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-4d1f6bb2-b8c5-4409-870c-f02c5cd1fa0e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5f922138cb74e89af084186fc0af166'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:59:49.344780 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:59:49.375755 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-4d1f6bb2-b8c5-4409-870c-f02c5cd1fa0e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5f922138cb74e89af084186fc0af166'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:59:49.392425 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 872/1747] 10.222.0.64 () {58 vars in 1224 bytes} [Tue Aug 2 09:59:49 2022] DELETE /identity/v3/users/f5f922138cb74e89af084186fc0af166 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:59:49.395590 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1453ef4-6f84-4b91-a64a-340a320f2742 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:49.397418 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b32554-7659-4b50-9918-ef2dfd0c1b6a None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:49.399486 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b32554-7659-4b50-9918-ef2dfd0c1b6a None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:49.399982 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b32554-7659-4b50-9918-ef2dfd0c1b6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:49.463142 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-e4b32554-7659-4b50-9918-ef2dfd0c1b6a None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:59:49.465194 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-e4b32554-7659-4b50-9918-ef2dfd0c1b6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:49.623378 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 873/1748] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 09:59:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:59:49.676398 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b32554-7659-4b50-9918-ef2dfd0c1b6a None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:49.677817 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-e4b32554-7659-4b50-9918-ef2dfd0c1b6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:49.678351 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2643de2-7293-42d8-a56c-f5e71914446b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:49.683143 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2643de2-7293-42d8-a56c-f5e71914446b admin admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:49.683556 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2643de2-7293-42d8-a56c-f5e71914446b admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:49.683737 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2643de2-7293-42d8-a56c-f5e71914446b admin admin] PATH_INFO: `/v3/projects/d186d210cff246749a7be936020e35d3/users/b8cbc6793d4f4554b4f364dbf45062d8/roles/66c1074780954f8e87ed11a5c4e8c36a` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:49.741868 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2643de2-7293-42d8-a56c-f5e71914446b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d186d210cff246749a7be936020e35d3, user_id=b8cbc6793d4f4554b4f364dbf45062d8, role_id=66c1074780954f8e87ed11a5c4e8c36a)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:49.749675 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2643de2-7293-42d8-a56c-f5e71914446b admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:49.799035 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:59:49.823107 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 876/1749] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 09:59:49 2022] PUT /identity/v3/projects/d186d210cff246749a7be936020e35d3/users/b8cbc6793d4f4554b4f364dbf45062d8/roles/66c1074780954f8e87ed11a5c4e8c36a => generated 0 bytes in 666 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:59:49.838528 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3635fe3-f438-4ad8-a32e-47226b227292 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:49.839171 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3635fe3-f438-4ad8-a32e-47226b227292 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:49.839678 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3635fe3-f438-4ad8-a32e-47226b227292 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:49.870235 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b32554-7659-4b50-9918-ef2dfd0c1b6a None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:49.947397 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-e3635fe3-f438-4ad8-a32e-47226b227292 None None] MFA Rules not processed for user `b8cbc6793d4f4554b4f364dbf45062d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:59:49.951811 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-e3635fe3-f438-4ad8-a32e-47226b227292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:50.121797 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686eef4e-e6c0-460a-86e2-8e0ce1e56b75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:50.122591 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686eef4e-e6c0-460a-86e2-8e0ce1e56b75 service neutron] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:50.122956 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686eef4e-e6c0-460a-86e2-8e0ce1e56b75 service neutron] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:50.123312 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686eef4e-e6c0-460a-86e2-8e0ce1e56b75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:50.175978 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 877/1750] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 09:59:49 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 340 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:59:50.181827 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3635fe3-f438-4ad8-a32e-47226b227292 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:50.187542 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-e3635fe3-f438-4ad8-a32e-47226b227292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:50.350018 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:59:50.368224 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686eef4e-e6c0-460a-86e2-8e0ce1e56b75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:50.372904 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686eef4e-e6c0-460a-86e2-8e0ce1e56b75 service neutron] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:50.393481 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 874/1751] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 09:59:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 717 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 09:59:50.396478 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3635fe3-f438-4ad8-a32e-47226b227292 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:50.416863 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cef0599-5c11-463d-aa17-c2c105b027d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:50.421313 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cef0599-5c11-463d-aa17-c2c105b027d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:50.422868 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cef0599-5c11-463d-aa17-c2c105b027d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:50.423901 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cef0599-5c11-463d-aa17-c2c105b027d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:50.426543 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cef0599-5c11-463d-aa17-c2c105b027d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:50.433461 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cef0599-5c11-463d-aa17-c2c105b027d7 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:50.521283 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a1e46b-fb43-4f41-a02e-a4ff74c4a57e None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:50.521734 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a1e46b-fb43-4f41-a02e-a4ff74c4a57e None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:50.521734 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a1e46b-fb43-4f41-a02e-a4ff74c4a57e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:50.542926 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 878/1752] 10.222.0.64 () {60 vars in 1153 bytes} [Tue Aug 2 09:59:50 2022] POST /identity/v3/projects => generated 369 bytes in 361 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:59:50.544783 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cef0599-5c11-463d-aa17-c2c105b027d7 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:50.639594 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-b7a1e46b-fb43-4f41-a02e-a4ff74c4a57e None None] MFA Rules not processed for user `b49f064508cb481e820933549826b257`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:59:50.641114 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-b7a1e46b-fb43-4f41-a02e-a4ff74c4a57e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:50.731171 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cef0599-5c11-463d-aa17-c2c105b027d7 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:50.746792 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91da94b3-caa5-4188-9d34-580ac9f92e18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:50.759856 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91da94b3-caa5-4188-9d34-580ac9f92e18 admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:50.760154 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91da94b3-caa5-4188-9d34-580ac9f92e18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:50.761914 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91da94b3-caa5-4188-9d34-580ac9f92e18 admin admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:50.764925 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91da94b3-caa5-4188-9d34-580ac9f92e18 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:50.771741 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91da94b3-caa5-4188-9d34-580ac9f92e18 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:50.866242 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 879/1753] 10.222.0.64 () {60 vars in 1144 bytes} [Tue Aug 2 09:59:50 2022] POST /identity/v3/users => generated 291 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:59:50.866242 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee3fdd6-3998-42c1-9644-69a9dd7af0c9 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:50.866824 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee3fdd6-3998-42c1-9644-69a9dd7af0c9 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:50.867443 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee3fdd6-3998-42c1-9644-69a9dd7af0c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:50.896493 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 875/1754] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 09:59:50 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 379 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:59:50.896723 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:59:50.896880 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a1e46b-fb43-4f41-a02e-a4ff74c4a57e None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:50.933527 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a1e46b-fb43-4f41-a02e-a4ff74c4a57e None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:50.960324 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fbb7a4-3525-430f-ba19-4a5465031212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:50.961224 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fbb7a4-3525-430f-ba19-4a5465031212 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:50.961648 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fbb7a4-3525-430f-ba19-4a5465031212 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:50.962020 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fbb7a4-3525-430f-ba19-4a5465031212 admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:50.963009 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fbb7a4-3525-430f-ba19-4a5465031212 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:50.966544 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-8ee3fdd6-3998-42c1-9644-69a9dd7af0c9 None None] MFA Rules not processed for user `c94f607a834f43e7a0073fe27d265901`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:59:50.972076 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-8ee3fdd6-3998-42c1-9644-69a9dd7af0c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:50.975783 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fbb7a4-3525-430f-ba19-4a5465031212 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:51.030916 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 876/1755] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 09:59:50 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:59:51.043913 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10601dc9-d2ef-446d-8e69-cadc5be35de5 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:51.047429 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10601dc9-d2ef-446d-8e69-cadc5be35de5 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:51.048430 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10601dc9-d2ef-446d-8e69-cadc5be35de5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:51.142364 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-10601dc9-d2ef-446d-8e69-cadc5be35de5 None None] MFA Rules not processed for user `b49f064508cb481e820933549826b257`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:59:51.144369 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-10601dc9-d2ef-446d-8e69-cadc5be35de5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:51.174736 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 877/1756] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 09:59:51 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:59:51.188330 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f7862a-99a4-4a6e-8903-7ce151706f22 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:51.189556 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f7862a-99a4-4a6e-8903-7ce151706f22 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:51.192344 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f7862a-99a4-4a6e-8903-7ce151706f22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:51.197647 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 880/1757] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 09:59:50 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 329 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:59:51.197647 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee3fdd6-3998-42c1-9644-69a9dd7af0c9 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:51.267817 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-16f7862a-99a4-4a6e-8903-7ce151706f22 None None] MFA Rules not processed for user `ec31b188970947efb2ed1d4d9a31cadf`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:59:51.272569 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-16f7862a-99a4-4a6e-8903-7ce151706f22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:51.318969 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee3fdd6-3998-42c1-9644-69a9dd7af0c9 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:51.356961 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6c25cb-7820-47e0-95b0-691713a32944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:51.360755 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6c25cb-7820-47e0-95b0-691713a32944 admin admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:51.361060 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6c25cb-7820-47e0-95b0-691713a32944 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:51.362947 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6c25cb-7820-47e0-95b0-691713a32944 admin admin] PATH_INFO: `/v3/projects/f0c4f9816e9049deb2eee40c058ecf33/users/9b0187b5b8cb4448a3dd7bf4f1a6048a/roles/1000cf318fa14915b8091599eaead5de` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:51.427782 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 878/1758] 10.222.0.83 () {58 vars in 1015 bytes} [Tue Aug 2 09:59:51 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:59:51.432397 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f7862a-99a4-4a6e-8903-7ce151706f22 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:51.436840 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-16f7862a-99a4-4a6e-8903-7ce151706f22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:51.458732 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6c25cb-7820-47e0-95b0-691713a32944 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0c4f9816e9049deb2eee40c058ecf33, user_id=9b0187b5b8cb4448a3dd7bf4f1a6048a, role_id=1000cf318fa14915b8091599eaead5de)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:51.464910 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6c25cb-7820-47e0-95b0-691713a32944 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:51.519773 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 881/1759] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 09:59:51 2022] PUT /identity/v3/projects/f0c4f9816e9049deb2eee40c058ecf33/users/9b0187b5b8cb4448a3dd7bf4f1a6048a/roles/1000cf318fa14915b8091599eaead5de => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:59:51.521167 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6c25cb-7820-47e0-95b0-691713a32944 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:51.523820 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-cf6c25cb-7820-47e0-95b0-691713a32944 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:51.649143 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6c25cb-7820-47e0-95b0-691713a32944 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:51.665958 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a105791-fb82-4cdd-a9f4-d73234b0a3e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:51.667062 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a105791-fb82-4cdd-a9f4-d73234b0a3e9 service neutron] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:51.667577 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a105791-fb82-4cdd-a9f4-d73234b0a3e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:51.668066 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a105791-fb82-4cdd-a9f4-d73234b0a3e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:51.685278 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a105791-fb82-4cdd-a9f4-d73234b0a3e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:51.687158 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a105791-fb82-4cdd-a9f4-d73234b0a3e9 service neutron] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:51.702431 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 882/1760] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 09:59:51 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:59:51.703691 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a105791-fb82-4cdd-a9f4-d73234b0a3e9 service neutron] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:51.704859 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f7862a-99a4-4a6e-8903-7ce151706f22 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:51.740795 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:59:51.836995 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a105791-fb82-4cdd-a9f4-d73234b0a3e9 service neutron] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:51.846969 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58092004-f0fa-43c1-9545-509cf9496f3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:51.847774 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58092004-f0fa-43c1-9545-509cf9496f3e admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:51.848165 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58092004-f0fa-43c1-9545-509cf9496f3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:51.848569 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58092004-f0fa-43c1-9545-509cf9496f3e admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:51.849398 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58092004-f0fa-43c1-9545-509cf9496f3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:51.851151 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58092004-f0fa-43c1-9545-509cf9496f3e admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:51.877676 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 883/1761] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 09:59:51 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:59:51.881523 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58092004-f0fa-43c1-9545-509cf9496f3e admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:51.905650 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58092004-f0fa-43c1-9545-509cf9496f3e admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:51.916991 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a719d9e-5fbb-412e-b141-018008c95053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:51.917930 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a719d9e-5fbb-412e-b141-018008c95053 admin admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:51.918228 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a719d9e-5fbb-412e-b141-018008c95053 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:51.918635 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a719d9e-5fbb-412e-b141-018008c95053 admin admin] PATH_INFO: `/v3/projects/f0c4f9816e9049deb2eee40c058ecf33/users/9b0187b5b8cb4448a3dd7bf4f1a6048a/roles/1000cf318fa14915b8091599eaead5de` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:51.921254 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a719d9e-5fbb-412e-b141-018008c95053 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0c4f9816e9049deb2eee40c058ecf33, user_id=9b0187b5b8cb4448a3dd7bf4f1a6048a, role_id=1000cf318fa14915b8091599eaead5de)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:51.925829 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a719d9e-5fbb-412e-b141-018008c95053 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:51.955806 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 884/1762] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 09:59:51 2022] PUT /identity/v3/projects/f0c4f9816e9049deb2eee40c058ecf33/users/9b0187b5b8cb4448a3dd7bf4f1a6048a/roles/1000cf318fa14915b8091599eaead5de => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:59:51.965866 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc76208f-23c4-48a2-8d29-d0405d62de8a None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:51.966304 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc76208f-23c4-48a2-8d29-d0405d62de8a None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:51.966678 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc76208f-23c4-48a2-8d29-d0405d62de8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:52.012094 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ad7c8d-89b1-4897-9e4a-3a70923a3fbb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99655e1f9ca24472a86cd26b0dbd1fc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:52.012712 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ad7c8d-89b1-4897-9e4a-3a70923a3fbb service nova] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:52.013601 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ad7c8d-89b1-4897-9e4a-3a70923a3fbb service nova] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:52.013866 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ad7c8d-89b1-4897-9e4a-3a70923a3fbb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:52.020618 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:59:52.030388 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-fc76208f-23c4-48a2-8d29-d0405d62de8a None None] MFA Rules not processed for user `9b0187b5b8cb4448a3dd7bf4f1a6048a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:59:52.034921 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-fc76208f-23c4-48a2-8d29-d0405d62de8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:52.177669 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ad7c8d-89b1-4897-9e4a-3a70923a3fbb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:52.179780 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ad7c8d-89b1-4897-9e4a-3a70923a3fbb service nova] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:52.191575 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 879/1763] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 09:59:51 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 763 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:59:52.192875 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ad7c8d-89b1-4897-9e4a-3a70923a3fbb service nova] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:52.194719 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-59ad7c8d-89b1-4897-9e4a-3a70923a3fbb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:52.199010 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 885/1764] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 09:59:51 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:59:52.199010 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc76208f-23c4-48a2-8d29-d0405d62de8a None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:52.199010 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-fc76208f-23c4-48a2-8d29-d0405d62de8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:52.237136 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ad7c8d-89b1-4897-9e4a-3a70923a3fbb service nova] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:52.390965 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d176a57-3939-4c91-b6ba-7bed6d2483e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:52.391732 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d176a57-3939-4c91-b6ba-7bed6d2483e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:52.392131 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d176a57-3939-4c91-b6ba-7bed6d2483e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:52.392822 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d176a57-3939-4c91-b6ba-7bed6d2483e1 admin admin] PATH_INFO: `/v3/projects/b87c162bd2a9412486babc73fa284607` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:52.393932 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d176a57-3939-4c91-b6ba-7bed6d2483e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b87c162bd2a9412486babc73fa284607)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:52.396294 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d176a57-3939-4c91-b6ba-7bed6d2483e1 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:52.403170 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc76208f-23c4-48a2-8d29-d0405d62de8a None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:52.428734 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ccc77e-bb23-4e13-9b92-c86cacd02852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:52.429966 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ccc77e-bb23-4e13-9b92-c86cacd02852 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:52.430644 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ccc77e-bb23-4e13-9b92-c86cacd02852 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:52.430644 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ccc77e-bb23-4e13-9b92-c86cacd02852 admin admin] PATH_INFO: `/v3/projects/94f6fb15f2e947c08d3f4fa12c0978c3` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:52.432037 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ccc77e-bb23-4e13-9b92-c86cacd02852 admin admin] RBAC: Authorizing `identity:delete_project(project_id=94f6fb15f2e947c08d3f4fa12c0978c3)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:52.437360 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ccc77e-bb23-4e13-9b92-c86cacd02852 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:52.579908 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-83ccc77e-bb23-4e13-9b92-c86cacd02852 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94f6fb15f2e947c08d3f4fa12c0978c3', 'request_id': 'req-83ccc77e-bb23-4e13-9b92-c86cacd02852'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:59:52.608269 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-83ccc77e-bb23-4e13-9b92-c86cacd02852 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94f6fb15f2e947c08d3f4fa12c0978c3', 'request_id': 'req-83ccc77e-bb23-4e13-9b92-c86cacd02852'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:59:52.618019 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-83ccc77e-bb23-4e13-9b92-c86cacd02852 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94f6fb15f2e947c08d3f4fa12c0978c3', 'request_id': 'req-83ccc77e-bb23-4e13-9b92-c86cacd02852'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:59:52.627258 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-7d176a57-3939-4c91-b6ba-7bed6d2483e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b87c162bd2a9412486babc73fa284607', 'request_id': 'req-7d176a57-3939-4c91-b6ba-7bed6d2483e1'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:59:52.633537 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:59:52.655632 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-83ccc77e-bb23-4e13-9b92-c86cacd02852 admin admin] The token cache is being invalidate because project 94f6fb15f2e947c08d3f4fa12c0978c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80153) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:59:52.655941 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-83ccc77e-bb23-4e13-9b92-c86cacd02852 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:59:52.657017 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-7d176a57-3939-4c91-b6ba-7bed6d2483e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b87c162bd2a9412486babc73fa284607', 'request_id': 'req-7d176a57-3939-4c91-b6ba-7bed6d2483e1'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:59:52.658709 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-83ccc77e-bb23-4e13-9b92-c86cacd02852 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:59:52.661592 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 886/1765] 10.222.0.64 () {58 vars in 1233 bytes} [Tue Aug 2 09:59:52 2022] DELETE /identity/v3/projects/94f6fb15f2e947c08d3f4fa12c0978c3 => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:59:52.666811 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ccc77e-bb23-4e13-9b92-c86cacd02852 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:52.669088 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-83ccc77e-bb23-4e13-9b92-c86cacd02852 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:52.674348 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-7d176a57-3939-4c91-b6ba-7bed6d2483e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b87c162bd2a9412486babc73fa284607', 'request_id': 'req-7d176a57-3939-4c91-b6ba-7bed6d2483e1'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:59:52.718413 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-7d176a57-3939-4c91-b6ba-7bed6d2483e1 admin admin] The token cache is being invalidate because project b87c162bd2a9412486babc73fa284607 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80154) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:59:52.718948 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-7d176a57-3939-4c91-b6ba-7bed6d2483e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:59:52.723146 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-7d176a57-3939-4c91-b6ba-7bed6d2483e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:59:52.726392 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 880/1766] 10.222.0.64 () {58 vars in 1233 bytes} [Tue Aug 2 09:59:52 2022] DELETE /identity/v3/projects/b87c162bd2a9412486babc73fa284607 => generated 0 bytes in 534 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:59:52.727304 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d176a57-3939-4c91-b6ba-7bed6d2483e1 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:52.733717 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-7d176a57-3939-4c91-b6ba-7bed6d2483e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:52.769763 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:59:52.864364 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ccc77e-bb23-4e13-9b92-c86cacd02852 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:52.888100 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1914067-6831-44b5-b11c-c1ea4747a2d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d7de62b9ee4039bdc79627ac7fb499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:52.888100 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1914067-6831-44b5-b11c-c1ea4747a2d9 service cinder] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:52.888100 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1914067-6831-44b5-b11c-c1ea4747a2d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:52.888100 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1914067-6831-44b5-b11c-c1ea4747a2d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:52.893745 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-f1914067-6831-44b5-b11c-c1ea4747a2d9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:52.977563 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d176a57-3939-4c91-b6ba-7bed6d2483e1 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:53.156862 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1914067-6831-44b5-b11c-c1ea4747a2d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:53.160406 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1914067-6831-44b5-b11c-c1ea4747a2d9 service cinder] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:53.178263 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 887/1767] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 09:59:52 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 513 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:59:53.181519 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66452cd7-c556-4c79-89b8-e436ed18280f None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:53.182046 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66452cd7-c556-4c79-89b8-e436ed18280f None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:53.182438 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66452cd7-c556-4c79-89b8-e436ed18280f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:53.225777 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-66452cd7-c556-4c79-89b8-e436ed18280f None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:59:53.228560 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-66452cd7-c556-4c79-89b8-e436ed18280f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:53.245064 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40bb27d-b51a-43ff-b457-cdc9e03d22df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:53.264267 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40bb27d-b51a-43ff-b457-cdc9e03d22df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:53.267905 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40bb27d-b51a-43ff-b457-cdc9e03d22df admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:53.268291 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40bb27d-b51a-43ff-b457-cdc9e03d22df admin admin] PATH_INFO: `/v3/users/9f843c14c3ef419ab4b05199bf131448` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:53.272234 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40bb27d-b51a-43ff-b457-cdc9e03d22df admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f843c14c3ef419ab4b05199bf131448)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:53.294088 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40bb27d-b51a-43ff-b457-cdc9e03d22df admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:53.435768 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 888/1768] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 09:59:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:59:53.436814 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-a40bb27d-b51a-43ff-b457-cdc9e03d22df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f843c14c3ef419ab4b05199bf131448'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:59:53.443854 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66452cd7-c556-4c79-89b8-e436ed18280f None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:53.455895 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-66452cd7-c556-4c79-89b8-e436ed18280f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:53.458326 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-a40bb27d-b51a-43ff-b457-cdc9e03d22df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f843c14c3ef419ab4b05199bf131448'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:59:53.459157 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-a40bb27d-b51a-43ff-b457-cdc9e03d22df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f843c14c3ef419ab4b05199bf131448'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:59:53.472747 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-a40bb27d-b51a-43ff-b457-cdc9e03d22df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f843c14c3ef419ab4b05199bf131448'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:59:53.490399 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66452cd7-c556-4c79-89b8-e436ed18280f None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:53.511786 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e2ddff-7458-4720-b17b-5dc756286174 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d7de62b9ee4039bdc79627ac7fb499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:53.512460 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e2ddff-7458-4720-b17b-5dc756286174 service cinder] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:53.512753 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e2ddff-7458-4720-b17b-5dc756286174 service cinder] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:53.513133 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e2ddff-7458-4720-b17b-5dc756286174 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:53.518749 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-f5e2ddff-7458-4720-b17b-5dc756286174 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:53.536290 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-a40bb27d-b51a-43ff-b457-cdc9e03d22df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f843c14c3ef419ab4b05199bf131448'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:59:53.563549 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 881/1769] 10.222.0.64 () {58 vars in 1224 bytes} [Tue Aug 2 09:59:52 2022] DELETE /identity/v3/users/9f843c14c3ef419ab4b05199bf131448 => generated 0 bytes in 836 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:59:53.567932 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce89fe4-374f-4ea9-b447-f77e135a4241 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:53.568297 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce89fe4-374f-4ea9-b447-f77e135a4241 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:53.568552 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce89fe4-374f-4ea9-b447-f77e135a4241 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:53.621341 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:59:53.650674 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-3ce89fe4-374f-4ea9-b447-f77e135a4241 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:59:53.657551 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-3ce89fe4-374f-4ea9-b447-f77e135a4241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:53.705488 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e2ddff-7458-4720-b17b-5dc756286174 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:53.706993 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e2ddff-7458-4720-b17b-5dc756286174 service cinder] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:53.739680 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 889/1770] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 09:59:53 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:59:53.869990 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 882/1771] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 09:59:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:59:53.874471 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e2ddff-7458-4720-b17b-5dc756286174 service cinder] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:53.887675 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e2ddff-7458-4720-b17b-5dc756286174 service cinder] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:53.918263 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e55436-71dd-4bab-9fdc-ac405178c866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:53.919376 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e55436-71dd-4bab-9fdc-ac405178c866 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:53.919665 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e55436-71dd-4bab-9fdc-ac405178c866 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:53.919946 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e55436-71dd-4bab-9fdc-ac405178c866 admin admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:53.920724 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e55436-71dd-4bab-9fdc-ac405178c866 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:59:53.921008 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e55436-71dd-4bab-9fdc-ac405178c866 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:53.922665 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e55436-71dd-4bab-9fdc-ac405178c866 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:53.975322 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce89fe4-374f-4ea9-b447-f77e135a4241 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:53.990544 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce89fe4-374f-4ea9-b447-f77e135a4241 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:53.991603 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 890/1772] 10.222.0.64 () {58 vars in 1153 bytes} [Tue Aug 2 09:59:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:59:54.005842 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5005aee-3a6a-4c3d-a692-1e767d9434dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:54.007058 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5005aee-3a6a-4c3d-a692-1e767d9434dc admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:54.007058 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5005aee-3a6a-4c3d-a692-1e767d9434dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:54.007058 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5005aee-3a6a-4c3d-a692-1e767d9434dc admin admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:54.007849 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5005aee-3a6a-4c3d-a692-1e767d9434dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:54.010954 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5005aee-3a6a-4c3d-a692-1e767d9434dc admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:54.126505 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e55436-71dd-4bab-9fdc-ac405178c866 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:54.127604 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 883/1773] 10.222.0.64 () {60 vars in 1153 bytes} [Tue Aug 2 09:59:53 2022] POST /identity/v3/projects => generated 377 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:59:54.132114 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:59:54.146728 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e55436-71dd-4bab-9fdc-ac405178c866 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:54.262390 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699c2994-c19e-4138-bfbe-5134cf4caa0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:54.263467 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699c2994-c19e-4138-bfbe-5134cf4caa0e admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:54.263559 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699c2994-c19e-4138-bfbe-5134cf4caa0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:54.263855 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699c2994-c19e-4138-bfbe-5134cf4caa0e admin admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:54.264526 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699c2994-c19e-4138-bfbe-5134cf4caa0e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:54.272311 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699c2994-c19e-4138-bfbe-5134cf4caa0e admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:54.322608 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5005aee-3a6a-4c3d-a692-1e767d9434dc admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:54.323540 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 891/1774] 10.222.0.64 () {60 vars in 1144 bytes} [Tue Aug 2 09:59:54 2022] POST /identity/v3/users => generated 289 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 09:59:54.336891 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5005aee-3a6a-4c3d-a692-1e767d9434dc admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:54.352522 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49beeeca-eb0c-47fc-8607-f872638fd19e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:54.353183 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49beeeca-eb0c-47fc-8607-f872638fd19e admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:54.353604 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49beeeca-eb0c-47fc-8607-f872638fd19e admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:54.353893 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49beeeca-eb0c-47fc-8607-f872638fd19e admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:54.354645 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49beeeca-eb0c-47fc-8607-f872638fd19e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:54.356324 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49beeeca-eb0c-47fc-8607-f872638fd19e admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:54.389210 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699c2994-c19e-4138-bfbe-5134cf4caa0e admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:54.399798 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699c2994-c19e-4138-bfbe-5134cf4caa0e admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:54.402644 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 884/1775] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 09:59:54 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:59:54.423248 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63d7e2a-fa60-4227-aacc-1dd9b252ae2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:54.423901 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63d7e2a-fa60-4227-aacc-1dd9b252ae2f admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:54.424197 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63d7e2a-fa60-4227-aacc-1dd9b252ae2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:54.424505 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63d7e2a-fa60-4227-aacc-1dd9b252ae2f admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:54.425232 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63d7e2a-fa60-4227-aacc-1dd9b252ae2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:54.426863 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63d7e2a-fa60-4227-aacc-1dd9b252ae2f admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:54.451291 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 892/1776] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 09:59:54 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 09:59:54.456201 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49beeeca-eb0c-47fc-8607-f872638fd19e admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:54.470887 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49beeeca-eb0c-47fc-8607-f872638fd19e admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:54.486823 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2945df-e010-4f27-858f-74a580666685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:54.487524 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2945df-e010-4f27-858f-74a580666685 admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:54.487817 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2945df-e010-4f27-858f-74a580666685 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:54.488102 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2945df-e010-4f27-858f-74a580666685 admin admin] PATH_INFO: `/v3/projects/c3b8c6297eea426cb29708e0ef857f47/users/9d15622994074a699eacc36c6b7d7123/roles/66c1074780954f8e87ed11a5c4e8c36a` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:54.515820 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2945df-e010-4f27-858f-74a580666685 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3b8c6297eea426cb29708e0ef857f47, user_id=9d15622994074a699eacc36c6b7d7123, role_id=66c1074780954f8e87ed11a5c4e8c36a)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:54.518628 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2945df-e010-4f27-858f-74a580666685 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:54.561797 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 885/1777] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 09:59:54 2022] PUT /identity/v3/projects/c3b8c6297eea426cb29708e0ef857f47/users/9d15622994074a699eacc36c6b7d7123/roles/66c1074780954f8e87ed11a5c4e8c36a => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:59:54.563405 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0f202d-5b3e-48b0-977a-a2744db2266a None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:54.563705 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0f202d-5b3e-48b0-977a-a2744db2266a None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:54.564003 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0f202d-5b3e-48b0-977a-a2744db2266a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:54.639399 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-ab0f202d-5b3e-48b0-977a-a2744db2266a None None] MFA Rules not processed for user `9d15622994074a699eacc36c6b7d7123`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:59:54.640823 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-ab0f202d-5b3e-48b0-977a-a2744db2266a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:54.646278 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9663d5f9-faf9-4b4f-9d23-011f64cd7871 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:54.646561 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9663d5f9-faf9-4b4f-9d23-011f64cd7871 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:54.646848 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9663d5f9-faf9-4b4f-9d23-011f64cd7871 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:54.650184 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 886/1778] 10.222.0.64 () {54 vars in 912 bytes} [Tue Aug 2 09:59:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:59:54.665306 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cf1aa7-2fba-4712-8c05-617e5687f07a None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:54.665729 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cf1aa7-2fba-4712-8c05-617e5687f07a None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:54.665873 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cf1aa7-2fba-4712-8c05-617e5687f07a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:54.738560 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-d9cf1aa7-2fba-4712-8c05-617e5687f07a None None] MFA Rules not processed for user `ec31b188970947efb2ed1d4d9a31cadf`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:59:54.743137 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-d9cf1aa7-2fba-4712-8c05-617e5687f07a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:54.875811 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 893/1779] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 09:59:54 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:59:54.882356 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0f202d-5b3e-48b0-977a-a2744db2266a None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:54.909274 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 887/1780] 10.222.0.64 () {58 vars in 1009 bytes} [Tue Aug 2 09:59:54 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Aug 02 09:59:55.076858 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0f202d-5b3e-48b0-977a-a2744db2266a None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:55.096888 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c48ee6-7129-4eb8-bdbb-8043ae56002d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:55.097799 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c48ee6-7129-4eb8-bdbb-8043ae56002d admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:55.098267 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c48ee6-7129-4eb8-bdbb-8043ae56002d admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:55.098695 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c48ee6-7129-4eb8-bdbb-8043ae56002d admin admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:55.099537 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c48ee6-7129-4eb8-bdbb-8043ae56002d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:55.101983 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c48ee6-7129-4eb8-bdbb-8043ae56002d admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:55.138936 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:59:55.175515 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cf1aa7-2fba-4712-8c05-617e5687f07a None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:55.183131 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 894/1781] 10.222.0.64 () {60 vars in 1153 bytes} [Tue Aug 2 09:59:54 2022] POST /identity/v3/projects => generated 375 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:59:55.226566 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cf1aa7-2fba-4712-8c05-617e5687f07a None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:55.379276 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c48ee6-7129-4eb8-bdbb-8043ae56002d admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:55.380925 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-56c48ee6-7129-4eb8-bdbb-8043ae56002d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:55.443684 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba00d86-0761-46fa-a5e6-aa261614e907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:55.444369 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba00d86-0761-46fa-a5e6-aa261614e907 admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:55.445656 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba00d86-0761-46fa-a5e6-aa261614e907 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:55.445943 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba00d86-0761-46fa-a5e6-aa261614e907 admin admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:55.449799 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba00d86-0761-46fa-a5e6-aa261614e907 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:55.456540 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba00d86-0761-46fa-a5e6-aa261614e907 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:55.490544 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:59:55.535696 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 888/1782] 10.222.0.64 () {60 vars in 1144 bytes} [Tue Aug 2 09:59:55 2022] POST /identity/v3/users => generated 294 bytes in 360 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:59:55.536428 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba00d86-0761-46fa-a5e6-aa261614e907 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:55.588664 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba00d86-0761-46fa-a5e6-aa261614e907 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:55.592737 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c48ee6-7129-4eb8-bdbb-8043ae56002d admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:55.601591 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199f8f80-ce8c-4976-911e-467d17eca00d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:55.602386 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199f8f80-ce8c-4976-911e-467d17eca00d admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:55.602670 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199f8f80-ce8c-4976-911e-467d17eca00d admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:55.602959 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199f8f80-ce8c-4976-911e-467d17eca00d admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:55.603814 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199f8f80-ce8c-4976-911e-467d17eca00d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:55.606582 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea4909c-a2cb-4ac1-8fe3-0329525c7e56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:55.607676 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea4909c-a2cb-4ac1-8fe3-0329525c7e56 service neutron] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:55.607964 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea4909c-a2cb-4ac1-8fe3-0329525c7e56 service neutron] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:55.608496 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea4909c-a2cb-4ac1-8fe3-0329525c7e56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:55.609426 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199f8f80-ce8c-4976-911e-467d17eca00d admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:55.624151 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea4909c-a2cb-4ac1-8fe3-0329525c7e56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:55.628350 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea4909c-a2cb-4ac1-8fe3-0329525c7e56 service neutron] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:55.641632 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 889/1783] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 09:59:55 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:59:55.642605 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199f8f80-ce8c-4976-911e-467d17eca00d admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:55.654075 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 895/1784] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 09:59:55 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 09:59:55.657482 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199f8f80-ce8c-4976-911e-467d17eca00d admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:55.663286 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2565823d-df85-4979-915b-d467b1b088a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:55.663979 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2565823d-df85-4979-915b-d467b1b088a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:55.664307 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2565823d-df85-4979-915b-d467b1b088a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:55.664592 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2565823d-df85-4979-915b-d467b1b088a0 admin admin] PATH_INFO: `/v3/projects/3bf9484227274dcbb4db19c776dd6ad5/users/6f20fd3be74e4702b366a1a00a77103f/roles/1000cf318fa14915b8091599eaead5de` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:55.692430 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2565823d-df85-4979-915b-d467b1b088a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bf9484227274dcbb4db19c776dd6ad5, user_id=6f20fd3be74e4702b366a1a00a77103f, role_id=1000cf318fa14915b8091599eaead5de)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:55.694948 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2565823d-df85-4979-915b-d467b1b088a0 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:55.725043 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 890/1785] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 09:59:55 2022] PUT /identity/v3/projects/3bf9484227274dcbb4db19c776dd6ad5/users/6f20fd3be74e4702b366a1a00a77103f/roles/1000cf318fa14915b8091599eaead5de => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:59:55.729390 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea4909c-a2cb-4ac1-8fe3-0329525c7e56 service neutron] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:55.750109 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea4909c-a2cb-4ac1-8fe3-0329525c7e56 service neutron] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:55.890660 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b099579f-a15f-4605-a9fd-e19c37950acf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:55.891392 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b099579f-a15f-4605-a9fd-e19c37950acf admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:55.891770 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b099579f-a15f-4605-a9fd-e19c37950acf admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:55.892155 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b099579f-a15f-4605-a9fd-e19c37950acf admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:55.893614 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b099579f-a15f-4605-a9fd-e19c37950acf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:55.895183 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b099579f-a15f-4605-a9fd-e19c37950acf admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:55.926253 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2565823d-df85-4979-915b-d467b1b088a0 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:55.926836 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 896/1786] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 09:59:55 2022] GET /identity/v3/roles => generated 1420 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:59:55.944418 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2565823d-df85-4979-915b-d467b1b088a0 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:55.967360 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551a2473-0512-460d-bfd7-2d79592b923d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:55.967691 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551a2473-0512-460d-bfd7-2d79592b923d admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:55.968052 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551a2473-0512-460d-bfd7-2d79592b923d admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:55.969088 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551a2473-0512-460d-bfd7-2d79592b923d admin admin] PATH_INFO: `/v3/projects/3bf9484227274dcbb4db19c776dd6ad5/users/6f20fd3be74e4702b366a1a00a77103f/roles/1000cf318fa14915b8091599eaead5de` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:55.976602 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551a2473-0512-460d-bfd7-2d79592b923d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bf9484227274dcbb4db19c776dd6ad5, user_id=6f20fd3be74e4702b366a1a00a77103f, role_id=1000cf318fa14915b8091599eaead5de)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:55.979983 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551a2473-0512-460d-bfd7-2d79592b923d admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:56.024821 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 891/1787] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 09:59:55 2022] PUT /identity/v3/projects/3bf9484227274dcbb4db19c776dd6ad5/users/6f20fd3be74e4702b366a1a00a77103f/roles/1000cf318fa14915b8091599eaead5de => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:59:56.039193 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404817a1-d494-4a67-9e2a-72ee2876db42 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:56.039491 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404817a1-d494-4a67-9e2a-72ee2876db42 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:56.039773 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404817a1-d494-4a67-9e2a-72ee2876db42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:56.079803 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-404817a1-d494-4a67-9e2a-72ee2876db42 None None] MFA Rules not processed for user `6f20fd3be74e4702b366a1a00a77103f`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:59:56.081739 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-404817a1-d494-4a67-9e2a-72ee2876db42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:56.219658 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 897/1788] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 09:59:56 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:59:56.241389 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551a2473-0512-460d-bfd7-2d79592b923d admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:56.243341 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-551a2473-0512-460d-bfd7-2d79592b923d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:56.304774 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ede22b1-162e-4029-887a-a86d44849e71 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:56.305209 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ede22b1-162e-4029-887a-a86d44849e71 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:56.305603 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ede22b1-162e-4029-887a-a86d44849e71 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:56.307776 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 898/1789] 10.222.0.64 () {54 vars in 921 bytes} [Tue Aug 2 09:59:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:59:56.347003 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe0559b-9876-451d-a79e-53f8d29a106d None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:56.347301 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe0559b-9876-451d-a79e-53f8d29a106d None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:56.347588 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe0559b-9876-451d-a79e-53f8d29a106d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:56.387702 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-ffe0559b-9876-451d-a79e-53f8d29a106d None None] MFA Rules not processed for user `99655e1f9ca24472a86cd26b0dbd1fc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:59:56.388700 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-ffe0559b-9876-451d-a79e-53f8d29a106d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:56.411779 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551a2473-0512-460d-bfd7-2d79592b923d admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:56.428346 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31f48a5-0c4e-4d77-ac8e-737483a5f27e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d7de62b9ee4039bdc79627ac7fb499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:56.428346 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31f48a5-0c4e-4d77-ac8e-737483a5f27e service cinder] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:56.428567 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31f48a5-0c4e-4d77-ac8e-737483a5f27e service cinder] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:56.429533 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31f48a5-0c4e-4d77-ac8e-737483a5f27e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:56.431580 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:59:56.582097 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 899/1790] 10.222.0.64 () {58 vars in 1018 bytes} [Tue Aug 2 09:59:56 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:59:56.713356 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31f48a5-0c4e-4d77-ac8e-737483a5f27e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:56.713356 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31f48a5-0c4e-4d77-ac8e-737483a5f27e service cinder] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:56.768286 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 892/1791] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 09:59:56 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 526 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:59:56.843095 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd14c22-c02b-48a0-9403-b5feaa44b47f None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:56.843283 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd14c22-c02b-48a0-9403-b5feaa44b47f None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:56.843745 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd14c22-c02b-48a0-9403-b5feaa44b47f None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:56.845826 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 900/1792] 10.222.0.64 () {54 vars in 921 bytes} [Tue Aug 2 09:59:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:59:56.951111 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8b90df-258b-4a25-8afb-f4829934435f None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:56.955759 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8b90df-258b-4a25-8afb-f4829934435f None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:56.955759 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8b90df-258b-4a25-8afb-f4829934435f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:56.993639 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-ea8b90df-258b-4a25-8afb-f4829934435f None None] MFA Rules not processed for user `99655e1f9ca24472a86cd26b0dbd1fc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:59:56.995104 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-ea8b90df-258b-4a25-8afb-f4829934435f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:57.105068 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 893/1793] 10.222.0.64 () {58 vars in 1018 bytes} [Tue Aug 2 09:59:56 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:59:57.974027 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd14c22-c02b-48a0-9403-b5feaa44b47f None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:57.976066 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-3bd14c22-c02b-48a0-9403-b5feaa44b47f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:58.164988 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd14c22-c02b-48a0-9403-b5feaa44b47f None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:58.177951 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbb4a9c-2e6a-4074-9805-391d13dd94f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:58.178976 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbb4a9c-2e6a-4074-9805-391d13dd94f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:58.179493 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbb4a9c-2e6a-4074-9805-391d13dd94f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:58.180048 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbb4a9c-2e6a-4074-9805-391d13dd94f5 admin admin] PATH_INFO: `/v3/projects/7684bbebb0184c2f99fe4437a63207c3` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:58.182440 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbb4a9c-2e6a-4074-9805-391d13dd94f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7684bbebb0184c2f99fe4437a63207c3)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:58.186941 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbb4a9c-2e6a-4074-9805-391d13dd94f5 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:58.343445 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-1dbb4a9c-2e6a-4074-9805-391d13dd94f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7684bbebb0184c2f99fe4437a63207c3', 'request_id': 'req-1dbb4a9c-2e6a-4074-9805-391d13dd94f5'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:59:58.362949 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-1dbb4a9c-2e6a-4074-9805-391d13dd94f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7684bbebb0184c2f99fe4437a63207c3', 'request_id': 'req-1dbb4a9c-2e6a-4074-9805-391d13dd94f5'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:59:58.378895 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-1dbb4a9c-2e6a-4074-9805-391d13dd94f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7684bbebb0184c2f99fe4437a63207c3', 'request_id': 'req-1dbb4a9c-2e6a-4074-9805-391d13dd94f5'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:59:58.413001 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-1dbb4a9c-2e6a-4074-9805-391d13dd94f5 admin admin] The token cache is being invalidate because project 7684bbebb0184c2f99fe4437a63207c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80153) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:59:58.413001 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-1dbb4a9c-2e6a-4074-9805-391d13dd94f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:59:58.414327 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-1dbb4a9c-2e6a-4074-9805-391d13dd94f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:59:58.427297 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8627a29-6715-43bc-9632-455091e1f444 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:58.427851 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8627a29-6715-43bc-9632-455091e1f444 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:58.427851 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8627a29-6715-43bc-9632-455091e1f444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:58.431379 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 901/1794] 10.222.0.64 () {58 vars in 1233 bytes} [Tue Aug 2 09:59:57 2022] DELETE /identity/v3/projects/7684bbebb0184c2f99fe4437a63207c3 => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:59:58.504739 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-b8627a29-6715-43bc-9632-455091e1f444 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:59:58.513676 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-b8627a29-6715-43bc-9632-455091e1f444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:58.725195 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:59:58.739360 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbb4a9c-2e6a-4074-9805-391d13dd94f5 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:58.742471 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-1dbb4a9c-2e6a-4074-9805-391d13dd94f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:58.795113 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 894/1795] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 09:59:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:59:58.809474 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e19f52-78bd-4466-838b-4546503374d8 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:58.809824 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e19f52-78bd-4466-838b-4546503374d8 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:58.813129 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e19f52-78bd-4466-838b-4546503374d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:58.924225 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-47e19f52-78bd-4466-838b-4546503374d8 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:59:58.932585 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-47e19f52-78bd-4466-838b-4546503374d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:58.984944 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:59:58.994969 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 895/1796] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 09:59:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:59:59.003102 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e19f52-78bd-4466-838b-4546503374d8 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:59.049154 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e19f52-78bd-4466-838b-4546503374d8 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:59.093281 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8d5cf2-8846-4779-baba-6d737e296a66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:59.098488 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8d5cf2-8846-4779-baba-6d737e296a66 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:59.098895 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8d5cf2-8846-4779-baba-6d737e296a66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:59.103528 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8d5cf2-8846-4779-baba-6d737e296a66 admin admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:59.104434 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8d5cf2-8846-4779-baba-6d737e296a66 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:59:59.104794 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8d5cf2-8846-4779-baba-6d737e296a66 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:59.119714 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8d5cf2-8846-4779-baba-6d737e296a66 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:59.150210 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbb4a9c-2e6a-4074-9805-391d13dd94f5 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:59.175756 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 896/1797] 10.222.0.64 () {58 vars in 1153 bytes} [Tue Aug 2 09:59:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:59:59.177674 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a5aa17-a60c-4dcb-ae95-e701d05130cd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99655e1f9ca24472a86cd26b0dbd1fc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:59.178409 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a5aa17-a60c-4dcb-ae95-e701d05130cd service nova] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:59.181216 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8d5cf2-8846-4779-baba-6d737e296a66 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:59.182451 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a5aa17-a60c-4dcb-ae95-e701d05130cd service nova] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:59.183343 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a5aa17-a60c-4dcb-ae95-e701d05130cd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:59.191699 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-f8a5aa17-a60c-4dcb-ae95-e701d05130cd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:59.279536 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8d5cf2-8846-4779-baba-6d737e296a66 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:59.293295 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140b8725-3c85-40ec-b352-c5230a6bdff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:59.294731 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140b8725-3c85-40ec-b352-c5230a6bdff2 admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:59.295707 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a5aa17-a60c-4dcb-ae95-e701d05130cd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:59.297101 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140b8725-3c85-40ec-b352-c5230a6bdff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:59.297553 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140b8725-3c85-40ec-b352-c5230a6bdff2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:59.299232 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140b8725-3c85-40ec-b352-c5230a6bdff2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:59.300689 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a5aa17-a60c-4dcb-ae95-e701d05130cd service nova] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:59.305026 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140b8725-3c85-40ec-b352-c5230a6bdff2 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:59.331757 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 902/1798] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 09:59:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 592 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:59:59.333184 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a5aa17-a60c-4dcb-ae95-e701d05130cd service nova] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:59.338187 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-f8a5aa17-a60c-4dcb-ae95-e701d05130cd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:59.371665 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a5aa17-a60c-4dcb-ae95-e701d05130cd service nova] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:59.390900 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 897/1799] 10.222.0.64 () {60 vars in 1153 bytes} [Tue Aug 2 09:59:59 2022] POST /identity/v3/projects => generated 365 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:59:59.392128 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140b8725-3c85-40ec-b352-c5230a6bdff2 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:59.405059 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f7ee06-ad61-4c6c-9563-b3f9e06fa080 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99655e1f9ca24472a86cd26b0dbd1fc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:59.407003 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f7ee06-ad61-4c6c-9563-b3f9e06fa080 service nova] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:59.407857 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f7ee06-ad61-4c6c-9563-b3f9e06fa080 service nova] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:59.408527 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f7ee06-ad61-4c6c-9563-b3f9e06fa080 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:59.413224 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-74f7ee06-ad61-4c6c-9563-b3f9e06fa080 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:59:59.452073 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f7ee06-ad61-4c6c-9563-b3f9e06fa080 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:59.454689 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f7ee06-ad61-4c6c-9563-b3f9e06fa080 service nova] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:59.482831 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 903/1800] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 09:59:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:59:59.576097 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140b8725-3c85-40ec-b352-c5230a6bdff2 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:59.593907 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568206be-3ea5-4318-928e-d48a0683e339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:59.594817 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568206be-3ea5-4318-928e-d48a0683e339 admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:59.595263 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568206be-3ea5-4318-928e-d48a0683e339 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:59.595709 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568206be-3ea5-4318-928e-d48a0683e339 admin admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:59.596985 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568206be-3ea5-4318-928e-d48a0683e339 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:59.599550 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568206be-3ea5-4318-928e-d48a0683e339 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:59.675367 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f7ee06-ad61-4c6c-9563-b3f9e06fa080 service nova] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:59.690095 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 898/1801] 10.222.0.64 () {60 vars in 1144 bytes} [Tue Aug 2 09:59:59 2022] POST /identity/v3/users => generated 283 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:59:59.698256 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f7ee06-ad61-4c6c-9563-b3f9e06fa080 service nova] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:59.710777 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507dd04a-297a-409f-8c4b-11641e68e2f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:59.711434 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507dd04a-297a-409f-8c4b-11641e68e2f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:59.711723 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507dd04a-297a-409f-8c4b-11641e68e2f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:59.712006 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507dd04a-297a-409f-8c4b-11641e68e2f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:59.712771 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507dd04a-297a-409f-8c4b-11641e68e2f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:59.714402 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507dd04a-297a-409f-8c4b-11641e68e2f7 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:59.731776 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568206be-3ea5-4318-928e-d48a0683e339 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:59.738258 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 904/1802] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 09:59:59 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:59:59.744386 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568206be-3ea5-4318-928e-d48a0683e339 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:59.750091 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f24e040-119f-436a-86b7-8c9bd3c85b0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:59.750745 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f24e040-119f-436a-86b7-8c9bd3c85b0b admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:59.751036 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f24e040-119f-436a-86b7-8c9bd3c85b0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:59.751312 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f24e040-119f-436a-86b7-8c9bd3c85b0b admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:59.752185 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f24e040-119f-436a-86b7-8c9bd3c85b0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:59.754027 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f24e040-119f-436a-86b7-8c9bd3c85b0b admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:59.771506 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 899/1803] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 09:59:59 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 09:59:59.775283 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507dd04a-297a-409f-8c4b-11641e68e2f7 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:59:59.791558 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507dd04a-297a-409f-8c4b-11641e68e2f7 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:59:59.799683 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9822ed66-549b-42f3-a442-4a69149a3a69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:59:59.801360 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9822ed66-549b-42f3-a442-4a69149a3a69 admin admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:59.802065 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9822ed66-549b-42f3-a442-4a69149a3a69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:59.802424 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9822ed66-549b-42f3-a442-4a69149a3a69 admin admin] PATH_INFO: `/v3/projects/7c802bb85c414be9ad026f527e5b2ffc/users/011a88cb815542e3ba499ac572f9553f/roles/66c1074780954f8e87ed11a5c4e8c36a` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:59.822397 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9822ed66-549b-42f3-a442-4a69149a3a69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c802bb85c414be9ad026f527e5b2ffc, user_id=011a88cb815542e3ba499ac572f9553f, role_id=66c1074780954f8e87ed11a5c4e8c36a)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:59:59.827154 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9822ed66-549b-42f3-a442-4a69149a3a69 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:59:59.862420 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 905/1804] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 09:59:59 2022] PUT /identity/v3/projects/7c802bb85c414be9ad026f527e5b2ffc/users/011a88cb815542e3ba499ac572f9553f/roles/66c1074780954f8e87ed11a5c4e8c36a => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:59:59.864024 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1147354c-21b3-4a7f-90f6-c28c5f42913d None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:59:59.864353 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1147354c-21b3-4a7f-90f6-c28c5f42913d None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:59:59.865841 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1147354c-21b3-4a7f-90f6-c28c5f42913d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:59:59.985312 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-1147354c-21b3-4a7f-90f6-c28c5f42913d None None] MFA Rules not processed for user `011a88cb815542e3ba499ac572f9553f`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:59:59.997563 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-1147354c-21b3-4a7f-90f6-c28c5f42913d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:00.154548 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980b3161-2811-49a6-9a1a-67479f3acc58 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:00.155023 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980b3161-2811-49a6-9a1a-67479f3acc58 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:00.155105 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980b3161-2811-49a6-9a1a-67479f3acc58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:00.269277 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-980b3161-2811-49a6-9a1a-67479f3acc58 None None] MFA Rules not processed for user `dfe32e33711e42118f3ff5021762604f`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:00:00.270621 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-980b3161-2811-49a6-9a1a-67479f3acc58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:00.278882 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 900/1805] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 09:59:59 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 415 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:00:00.280224 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1147354c-21b3-4a7f-90f6-c28c5f42913d None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:00.361633 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:00.433111 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 906/1806] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:00:00 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:00:00.537135 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1147354c-21b3-4a7f-90f6-c28c5f42913d None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:00.560489 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a580360d-3205-40b5-bc6a-76fdde4583f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:00.562173 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a580360d-3205-40b5-bc6a-76fdde4583f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:00.562477 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a580360d-3205-40b5-bc6a-76fdde4583f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:00.562769 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a580360d-3205-40b5-bc6a-76fdde4583f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:00.563482 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a580360d-3205-40b5-bc6a-76fdde4583f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:00.575959 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a580360d-3205-40b5-bc6a-76fdde4583f3 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:00.646320 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 901/1807] 10.222.0.64 () {60 vars in 1153 bytes} [Tue Aug 2 10:00:00 2022] POST /identity/v3/projects => generated 367 bytes in 367 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:00:00.648983 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980b3161-2811-49a6-9a1a-67479f3acc58 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:00.666840 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980b3161-2811-49a6-9a1a-67479f3acc58 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:00.754975 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a580360d-3205-40b5-bc6a-76fdde4583f3 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:00.757295 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-a580360d-3205-40b5-bc6a-76fdde4583f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:00.885415 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826bf1de-c421-4fd3-b3c8-7db824533a43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:00.886836 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826bf1de-c421-4fd3-b3c8-7db824533a43 admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:00.888857 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826bf1de-c421-4fd3-b3c8-7db824533a43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:00.895519 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826bf1de-c421-4fd3-b3c8-7db824533a43 admin admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:00.899377 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826bf1de-c421-4fd3-b3c8-7db824533a43 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:00.901975 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826bf1de-c421-4fd3-b3c8-7db824533a43 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:00.924565 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:00.965441 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 907/1808] 10.222.0.64 () {60 vars in 1144 bytes} [Tue Aug 2 10:00:00 2022] POST /identity/v3/users => generated 290 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:00:00.965441 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826bf1de-c421-4fd3-b3c8-7db824533a43 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:00.995845 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a580360d-3205-40b5-bc6a-76fdde4583f3 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:01.014855 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d00996-f95d-48d0-83f9-fae0655916aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:01.015466 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d00996-f95d-48d0-83f9-fae0655916aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:01.017645 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d00996-f95d-48d0-83f9-fae0655916aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:01.017949 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d00996-f95d-48d0-83f9-fae0655916aa admin admin] PATH_INFO: `/v3/users/b8cbc6793d4f4554b4f364dbf45062d8` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:01.022653 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d00996-f95d-48d0-83f9-fae0655916aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8cbc6793d4f4554b4f364dbf45062d8)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:01.032566 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d00996-f95d-48d0-83f9-fae0655916aa admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:01.226279 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826bf1de-c421-4fd3-b3c8-7db824533a43 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:01.235808 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725ec483-68e2-4c82-b97e-e67d6268aa51 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99655e1f9ca24472a86cd26b0dbd1fc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:01.237208 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725ec483-68e2-4c82-b97e-e67d6268aa51 service nova] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:01.237709 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725ec483-68e2-4c82-b97e-e67d6268aa51 service nova] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:01.238205 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725ec483-68e2-4c82-b97e-e67d6268aa51 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:01.254163 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-34d00996-f95d-48d0-83f9-fae0655916aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8cbc6793d4f4554b4f364dbf45062d8'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:01.275350 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-34d00996-f95d-48d0-83f9-fae0655916aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8cbc6793d4f4554b4f364dbf45062d8'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:01.276620 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-34d00996-f95d-48d0-83f9-fae0655916aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8cbc6793d4f4554b4f364dbf45062d8'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:01.303708 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-34d00996-f95d-48d0-83f9-fae0655916aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8cbc6793d4f4554b4f364dbf45062d8'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:01.334663 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-34d00996-f95d-48d0-83f9-fae0655916aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8cbc6793d4f4554b4f364dbf45062d8'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:01.360067 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 902/1809] 10.222.0.64 () {58 vars in 1224 bytes} [Tue Aug 2 10:00:00 2022] DELETE /identity/v3/users/b8cbc6793d4f4554b4f364dbf45062d8 => generated 0 bytes in 603 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:00:01.360067 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d00996-f95d-48d0-83f9-fae0655916aa admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:01.361575 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-34d00996-f95d-48d0-83f9-fae0655916aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:01.422650 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:01.430402 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725ec483-68e2-4c82-b97e-e67d6268aa51 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:01.432584 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725ec483-68e2-4c82-b97e-e67d6268aa51 service nova] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:01.447123 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 908/1810] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:00:00 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 484 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:00:01.458620 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26ce622-465a-4b9c-8556-ff1069d19970 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:01.458972 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26ce622-465a-4b9c-8556-ff1069d19970 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:01.459194 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26ce622-465a-4b9c-8556-ff1069d19970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:01.518499 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-c26ce622-465a-4b9c-8556-ff1069d19970 None None] MFA Rules not processed for user `982a01f0d3a5459a9add3f6f23db5db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:00:01.519677 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d00996-f95d-48d0-83f9-fae0655916aa admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:01.519806 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-c26ce622-465a-4b9c-8556-ff1069d19970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:01.539328 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25219716-43a2-497b-bdf4-742d1b976b6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:01.540094 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25219716-43a2-497b-bdf4-742d1b976b6d admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:01.541859 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25219716-43a2-497b-bdf4-742d1b976b6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:01.542254 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25219716-43a2-497b-bdf4-742d1b976b6d admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:01.545114 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25219716-43a2-497b-bdf4-742d1b976b6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:01.548458 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25219716-43a2-497b-bdf4-742d1b976b6d admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:01.585181 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 903/1811] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:00:01 2022] GET /identity/v3/roles => generated 1420 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:00:01.593771 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988149a3-b32b-440a-ab45-72f04222586b None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:01.594306 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988149a3-b32b-440a-ab45-72f04222586b None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:01.594771 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988149a3-b32b-440a-ab45-72f04222586b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:01.647763 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-988149a3-b32b-440a-ab45-72f04222586b None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:00:01.649112 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-988149a3-b32b-440a-ab45-72f04222586b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:01.671088 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 904/1812] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:00:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:00:01.676619 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988149a3-b32b-440a-ab45-72f04222586b None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:01.697131 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988149a3-b32b-440a-ab45-72f04222586b None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:01.702804 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d64d33-5be2-4d3b-8826-545e3940f44a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:01.704148 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d64d33-5be2-4d3b-8826-545e3940f44a admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:01.704876 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d64d33-5be2-4d3b-8826-545e3940f44a admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:01.705727 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d64d33-5be2-4d3b-8826-545e3940f44a admin admin] PATH_INFO: `/v3/projects/1a0bd282d8de482099bc32854206d3c6/users/97dbad29b78645f88895f22f7039fbc9/roles/1000cf318fa14915b8091599eaead5de` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:01.726972 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d64d33-5be2-4d3b-8826-545e3940f44a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a0bd282d8de482099bc32854206d3c6, user_id=97dbad29b78645f88895f22f7039fbc9, role_id=1000cf318fa14915b8091599eaead5de)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:01.731374 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d64d33-5be2-4d3b-8826-545e3940f44a admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:01.761728 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 909/1813] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:00:01 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:00:01.762735 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26ce622-465a-4b9c-8556-ff1069d19970 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:01.766478 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-c26ce622-465a-4b9c-8556-ff1069d19970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:01.779468 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 905/1814] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 10:00:01 2022] PUT /identity/v3/projects/1a0bd282d8de482099bc32854206d3c6/users/97dbad29b78645f88895f22f7039fbc9/roles/1000cf318fa14915b8091599eaead5de => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:00:01.785523 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d64d33-5be2-4d3b-8826-545e3940f44a admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:01.895027 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:01.926199 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26ce622-465a-4b9c-8556-ff1069d19970 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:02.021302 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fb3af6-1096-4719-b670-066992eb0451 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:02.022307 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fb3af6-1096-4719-b670-066992eb0451 service neutron] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:02.022797 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fb3af6-1096-4719-b670-066992eb0451 service neutron] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:02.023230 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fb3af6-1096-4719-b670-066992eb0451 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:02.082708 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d64d33-5be2-4d3b-8826-545e3940f44a admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:02.118115 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0625f1bb-017d-4599-b3ab-752e3168edbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:02.123639 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0625f1bb-017d-4599-b3ab-752e3168edbd admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:02.133191 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0625f1bb-017d-4599-b3ab-752e3168edbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:02.133735 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0625f1bb-017d-4599-b3ab-752e3168edbd admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:02.134668 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0625f1bb-017d-4599-b3ab-752e3168edbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:02.137088 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0625f1bb-017d-4599-b3ab-752e3168edbd admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:02.141269 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fb3af6-1096-4719-b670-066992eb0451 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:02.143092 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fb3af6-1096-4719-b670-066992eb0451 service neutron] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:02.157103 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 910/1815] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:00:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:00:02.159420 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 906/1816] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:00:01 2022] GET /identity/v3/roles => generated 1420 bytes in 376 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:00:02.166482 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fb3af6-1096-4719-b670-066992eb0451 service neutron] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:02.182283 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fb3af6-1096-4719-b670-066992eb0451 service neutron] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:02.191898 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa26ed9-fe32-4875-9b9d-bfa10ab1acb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:02.192885 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa26ed9-fe32-4875-9b9d-bfa10ab1acb3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:02.193281 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa26ed9-fe32-4875-9b9d-bfa10ab1acb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:02.194497 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa26ed9-fe32-4875-9b9d-bfa10ab1acb3 admin admin] PATH_INFO: `/v3/projects/1a0bd282d8de482099bc32854206d3c6/users/97dbad29b78645f88895f22f7039fbc9/roles/1000cf318fa14915b8091599eaead5de` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:02.201209 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa26ed9-fe32-4875-9b9d-bfa10ab1acb3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a0bd282d8de482099bc32854206d3c6, user_id=97dbad29b78645f88895f22f7039fbc9, role_id=1000cf318fa14915b8091599eaead5de)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:02.205141 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa26ed9-fe32-4875-9b9d-bfa10ab1acb3 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:02.234910 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89c7b7a-42c5-44fa-acf6-16eb27ce6a5f None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:02.234910 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89c7b7a-42c5-44fa-acf6-16eb27ce6a5f None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:02.234910 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89c7b7a-42c5-44fa-acf6-16eb27ce6a5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:02.235786 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 911/1817] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 10:00:02 2022] PUT /identity/v3/projects/1a0bd282d8de482099bc32854206d3c6/users/97dbad29b78645f88895f22f7039fbc9/roles/1000cf318fa14915b8091599eaead5de => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:00:02.287409 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-c89c7b7a-42c5-44fa-acf6-16eb27ce6a5f None None] MFA Rules not processed for user `97dbad29b78645f88895f22f7039fbc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:00:02.287409 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-c89c7b7a-42c5-44fa-acf6-16eb27ce6a5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:02.402151 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 907/1818] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:00:02 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:00:02.417330 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa26ed9-fe32-4875-9b9d-bfa10ab1acb3 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:02.421059 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-8fa26ed9-fe32-4875-9b9d-bfa10ab1acb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:02.596380 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa26ed9-fe32-4875-9b9d-bfa10ab1acb3 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:02.609790 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c3c95c-9d07-45c4-9cec-b3d4bbe569b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d7de62b9ee4039bdc79627ac7fb499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:02.612236 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c3c95c-9d07-45c4-9cec-b3d4bbe569b8 service cinder] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:02.612552 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c3c95c-9d07-45c4-9cec-b3d4bbe569b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:02.613528 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c3c95c-9d07-45c4-9cec-b3d4bbe569b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:02.637156 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c3c95c-9d07-45c4-9cec-b3d4bbe569b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:02.642471 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c3c95c-9d07-45c4-9cec-b3d4bbe569b8 service cinder] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:02.672371 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 912/1819] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:00:02 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:00:02.823110 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89c7b7a-42c5-44fa-acf6-16eb27ce6a5f None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:02.847490 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89c7b7a-42c5-44fa-acf6-16eb27ce6a5f None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:02.867735 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd829c5d-62ba-437e-b82f-b936109c9888 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d7de62b9ee4039bdc79627ac7fb499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:02.868310 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd829c5d-62ba-437e-b82f-b936109c9888 service cinder] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:02.868600 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd829c5d-62ba-437e-b82f-b936109c9888 service cinder] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:02.868876 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd829c5d-62ba-437e-b82f-b936109c9888 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:02.877985 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-fd829c5d-62ba-437e-b82f-b936109c9888 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:02.891609 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:03.010537 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd829c5d-62ba-437e-b82f-b936109c9888 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:03.012163 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd829c5d-62ba-437e-b82f-b936109c9888 service cinder] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:03.029318 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 908/1820] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:00:02 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:00:03.339255 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c3c95c-9d07-45c4-9cec-b3d4bbe569b8 service cinder] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:03.341845 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-e0c3c95c-9d07-45c4-9cec-b3d4bbe569b8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:03.464710 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:03.521419 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c3c95c-9d07-45c4-9cec-b3d4bbe569b8 service cinder] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:03.536440 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a638c083-a28c-4d47-b32d-47d1de3944d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:03.537111 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a638c083-a28c-4d47-b32d-47d1de3944d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:03.537396 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a638c083-a28c-4d47-b32d-47d1de3944d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:03.537672 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a638c083-a28c-4d47-b32d-47d1de3944d0 admin admin] PATH_INFO: `/v3/projects/d186d210cff246749a7be936020e35d3` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:03.539010 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a638c083-a28c-4d47-b32d-47d1de3944d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d186d210cff246749a7be936020e35d3)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:03.540989 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a638c083-a28c-4d47-b32d-47d1de3944d0 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:03.729260 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-a638c083-a28c-4d47-b32d-47d1de3944d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd186d210cff246749a7be936020e35d3', 'request_id': 'req-a638c083-a28c-4d47-b32d-47d1de3944d0'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:03.741063 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-a638c083-a28c-4d47-b32d-47d1de3944d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd186d210cff246749a7be936020e35d3', 'request_id': 'req-a638c083-a28c-4d47-b32d-47d1de3944d0'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:03.749722 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-a638c083-a28c-4d47-b32d-47d1de3944d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd186d210cff246749a7be936020e35d3', 'request_id': 'req-a638c083-a28c-4d47-b32d-47d1de3944d0'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:03.769978 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-a638c083-a28c-4d47-b32d-47d1de3944d0 admin admin] The token cache is being invalidate because project d186d210cff246749a7be936020e35d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80153) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:00:03.770346 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-a638c083-a28c-4d47-b32d-47d1de3944d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:03.772902 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-a638c083-a28c-4d47-b32d-47d1de3944d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:03.780139 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd829c5d-62ba-437e-b82f-b936109c9888 service cinder] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:03.780738 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 913/1821] 10.222.0.64 () {58 vars in 1233 bytes} [Tue Aug 2 10:00:03 2022] DELETE /identity/v3/projects/d186d210cff246749a7be936020e35d3 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:00:03.799715 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd829c5d-62ba-437e-b82f-b936109c9888 service cinder] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:03.892742 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467e004f-2056-4b1b-8e51-d03f7c879795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:03.894291 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467e004f-2056-4b1b-8e51-d03f7c879795 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:03.894850 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467e004f-2056-4b1b-8e51-d03f7c879795 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:03.895366 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467e004f-2056-4b1b-8e51-d03f7c879795 admin admin] PATH_INFO: `/v3/users/9b0187b5b8cb4448a3dd7bf4f1a6048a` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:03.897030 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467e004f-2056-4b1b-8e51-d03f7c879795 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b0187b5b8cb4448a3dd7bf4f1a6048a)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:03.899520 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467e004f-2056-4b1b-8e51-d03f7c879795 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:03.996021 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-467e004f-2056-4b1b-8e51-d03f7c879795 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b0187b5b8cb4448a3dd7bf4f1a6048a'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:04.015288 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-467e004f-2056-4b1b-8e51-d03f7c879795 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b0187b5b8cb4448a3dd7bf4f1a6048a'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:04.016595 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-467e004f-2056-4b1b-8e51-d03f7c879795 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b0187b5b8cb4448a3dd7bf4f1a6048a'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:04.032274 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-467e004f-2056-4b1b-8e51-d03f7c879795 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b0187b5b8cb4448a3dd7bf4f1a6048a'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:04.058645 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-467e004f-2056-4b1b-8e51-d03f7c879795 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b0187b5b8cb4448a3dd7bf4f1a6048a'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:04.067501 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 909/1822] 10.222.0.64 () {58 vars in 1224 bytes} [Tue Aug 2 10:00:03 2022] DELETE /identity/v3/users/9b0187b5b8cb4448a3dd7bf4f1a6048a => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:00:04.717184 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1087e78-a3d2-4c84-bee2-2669409375e3 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:04.717714 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1087e78-a3d2-4c84-bee2-2669409375e3 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:04.720387 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1087e78-a3d2-4c84-bee2-2669409375e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:04.744630 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467e004f-2056-4b1b-8e51-d03f7c879795 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:04.749161 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-467e004f-2056-4b1b-8e51-d03f7c879795 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:04.763867 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-e1087e78-a3d2-4c84-bee2-2669409375e3 None None] MFA Rules not processed for user `dfe32e33711e42118f3ff5021762604f`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:00:04.765579 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-e1087e78-a3d2-4c84-bee2-2669409375e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:04.806817 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:04.904847 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 914/1823] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:00:04 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:00:04.905919 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467e004f-2056-4b1b-8e51-d03f7c879795 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:04.910813 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e561ed-671b-419c-981c-2cac84d7b40d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:04.911681 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e561ed-671b-419c-981c-2cac84d7b40d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:04.912506 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e561ed-671b-419c-981c-2cac84d7b40d admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:04.915364 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e561ed-671b-419c-981c-2cac84d7b40d admin admin] PATH_INFO: `/v3/projects/f0c4f9816e9049deb2eee40c058ecf33` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:04.916668 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e561ed-671b-419c-981c-2cac84d7b40d admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0c4f9816e9049deb2eee40c058ecf33)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:04.918834 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e561ed-671b-419c-981c-2cac84d7b40d admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:05.014443 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-85e561ed-671b-419c-981c-2cac84d7b40d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0c4f9816e9049deb2eee40c058ecf33', 'request_id': 'req-85e561ed-671b-419c-981c-2cac84d7b40d'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:05.021461 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-85e561ed-671b-419c-981c-2cac84d7b40d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0c4f9816e9049deb2eee40c058ecf33', 'request_id': 'req-85e561ed-671b-419c-981c-2cac84d7b40d'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:05.030988 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-85e561ed-671b-419c-981c-2cac84d7b40d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0c4f9816e9049deb2eee40c058ecf33', 'request_id': 'req-85e561ed-671b-419c-981c-2cac84d7b40d'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:05.054429 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-85e561ed-671b-419c-981c-2cac84d7b40d admin admin] The token cache is being invalidate because project f0c4f9816e9049deb2eee40c058ecf33 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80154) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:00:05.055133 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-85e561ed-671b-419c-981c-2cac84d7b40d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:05.058510 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-85e561ed-671b-419c-981c-2cac84d7b40d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:05.065160 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 910/1824] 10.222.0.64 () {58 vars in 1233 bytes} [Tue Aug 2 10:00:04 2022] DELETE /identity/v3/projects/f0c4f9816e9049deb2eee40c058ecf33 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:00:05.066270 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a760d6b-270d-4c5e-b5e6-605ae9ffa625 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:05.066750 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a760d6b-270d-4c5e-b5e6-605ae9ffa625 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:05.067338 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a760d6b-270d-4c5e-b5e6-605ae9ffa625 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:05.094004 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-1a760d6b-270d-4c5e-b5e6-605ae9ffa625 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:00:05.096127 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-1a760d6b-270d-4c5e-b5e6-605ae9ffa625 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:05.207843 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ff8d56-ae98-44f9-baa9-61843a973a76 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:05.208142 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ff8d56-ae98-44f9-baa9-61843a973a76 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:05.208458 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ff8d56-ae98-44f9-baa9-61843a973a76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:05.212683 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 915/1825] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:00:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:00:05.245466 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-44ff8d56-ae98-44f9-baa9-61843a973a76 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:00:05.246781 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-44ff8d56-ae98-44f9-baa9-61843a973a76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:05.260695 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 911/1826] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:00:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:00:05.265578 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a760d6b-270d-4c5e-b5e6-605ae9ffa625 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:05.283297 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a760d6b-270d-4c5e-b5e6-605ae9ffa625 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:05.294107 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8281fe-0497-4d2b-9cb8-2e685695c84e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:05.295041 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8281fe-0497-4d2b-9cb8-2e685695c84e admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:05.295469 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8281fe-0497-4d2b-9cb8-2e685695c84e admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:05.296138 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8281fe-0497-4d2b-9cb8-2e685695c84e admin admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:05.297747 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8281fe-0497-4d2b-9cb8-2e685695c84e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:00:05.298464 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8281fe-0497-4d2b-9cb8-2e685695c84e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:05.300626 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8281fe-0497-4d2b-9cb8-2e685695c84e admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:05.310390 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:05.321016 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 916/1827] 10.222.0.64 () {58 vars in 1153 bytes} [Tue Aug 2 10:00:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:05.322408 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ff8d56-ae98-44f9-baa9-61843a973a76 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:05.336384 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ff8d56-ae98-44f9-baa9-61843a973a76 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:05.343201 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76436dd8-f82c-4b15-98c4-f98688209506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:05.343836 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76436dd8-f82c-4b15-98c4-f98688209506 admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:05.344154 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76436dd8-f82c-4b15-98c4-f98688209506 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:05.344630 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76436dd8-f82c-4b15-98c4-f98688209506 admin admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:05.345636 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76436dd8-f82c-4b15-98c4-f98688209506 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:05.350399 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76436dd8-f82c-4b15-98c4-f98688209506 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:05.391380 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 912/1828] 10.222.0.64 () {60 vars in 1153 bytes} [Tue Aug 2 10:00:05 2022] POST /identity/v3/projects => generated 401 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:00:05.391727 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8281fe-0497-4d2b-9cb8-2e685695c84e admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:05.403592 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8281fe-0497-4d2b-9cb8-2e685695c84e admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:05.484847 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c24cad5-705a-4fe9-a833-e01cf2217ba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:05.485486 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c24cad5-705a-4fe9-a833-e01cf2217ba1 admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:05.485775 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c24cad5-705a-4fe9-a833-e01cf2217ba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:05.486059 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c24cad5-705a-4fe9-a833-e01cf2217ba1 admin admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:05.486739 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c24cad5-705a-4fe9-a833-e01cf2217ba1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:05.488720 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c24cad5-705a-4fe9-a833-e01cf2217ba1 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:05.514712 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 917/1829] 10.222.0.64 () {60 vars in 1144 bytes} [Tue Aug 2 10:00:05 2022] POST /identity/v3/users => generated 301 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:00:05.518497 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76436dd8-f82c-4b15-98c4-f98688209506 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:05.529497 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76436dd8-f82c-4b15-98c4-f98688209506 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:05.536285 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a8347c-bfd4-4a73-964f-69e9b139edac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:05.537116 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a8347c-bfd4-4a73-964f-69e9b139edac admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:05.537406 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a8347c-bfd4-4a73-964f-69e9b139edac admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:05.537736 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a8347c-bfd4-4a73-964f-69e9b139edac admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:05.538431 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a8347c-bfd4-4a73-964f-69e9b139edac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:05.540039 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a8347c-bfd4-4a73-964f-69e9b139edac admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:05.568079 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 913/1830] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:00:05 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:00:05.575855 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c24cad5-705a-4fe9-a833-e01cf2217ba1 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:05.587928 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c24cad5-705a-4fe9-a833-e01cf2217ba1 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:05.593929 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd26f23-f432-455d-aead-6c4de9b534cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:05.594574 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd26f23-f432-455d-aead-6c4de9b534cb admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:05.594863 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd26f23-f432-455d-aead-6c4de9b534cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:05.595145 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd26f23-f432-455d-aead-6c4de9b534cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:05.595860 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd26f23-f432-455d-aead-6c4de9b534cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:05.597531 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd26f23-f432-455d-aead-6c4de9b534cb admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:05.612936 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 918/1831] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:00:05 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:00:05.614102 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a8347c-bfd4-4a73-964f-69e9b139edac admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:05.625141 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a8347c-bfd4-4a73-964f-69e9b139edac admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:05.629926 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55975067-2968-4057-b85e-26c9a9dcfad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:05.630613 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55975067-2968-4057-b85e-26c9a9dcfad0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:05.630903 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55975067-2968-4057-b85e-26c9a9dcfad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:05.631215 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55975067-2968-4057-b85e-26c9a9dcfad0 admin admin] PATH_INFO: `/v3/projects/ff68847a6af247d8a372053aa84500f2/users/39deafc81f694dd795de14b07e56b47b/roles/66c1074780954f8e87ed11a5c4e8c36a` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:05.651285 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55975067-2968-4057-b85e-26c9a9dcfad0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff68847a6af247d8a372053aa84500f2, user_id=39deafc81f694dd795de14b07e56b47b, role_id=66c1074780954f8e87ed11a5c4e8c36a)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:05.654104 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55975067-2968-4057-b85e-26c9a9dcfad0 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:05.672040 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 914/1832] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 10:00:05 2022] PUT /identity/v3/projects/ff68847a6af247d8a372053aa84500f2/users/39deafc81f694dd795de14b07e56b47b/roles/66c1074780954f8e87ed11a5c4e8c36a => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:00:05.683425 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c281100-ac50-4102-afe3-0bdabefa16ce None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:05.683722 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c281100-ac50-4102-afe3-0bdabefa16ce None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:05.684033 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c281100-ac50-4102-afe3-0bdabefa16ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:05.719097 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-3c281100-ac50-4102-afe3-0bdabefa16ce None None] MFA Rules not processed for user `39deafc81f694dd795de14b07e56b47b`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:00:05.720523 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-3c281100-ac50-4102-afe3-0bdabefa16ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:05.794678 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 919/1833] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:00:05 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:00:05.799584 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55975067-2968-4057-b85e-26c9a9dcfad0 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:05.889222 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:05.899699 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55975067-2968-4057-b85e-26c9a9dcfad0 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:05.906827 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e088ff0-f2bb-4be8-a32e-fbf06d0079a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:05.907384 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e088ff0-f2bb-4be8-a32e-fbf06d0079a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:05.907384 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e088ff0-f2bb-4be8-a32e-fbf06d0079a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:05.907805 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e088ff0-f2bb-4be8-a32e-fbf06d0079a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:05.908586 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e088ff0-f2bb-4be8-a32e-fbf06d0079a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:05.911332 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e088ff0-f2bb-4be8-a32e-fbf06d0079a8 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:05.961781 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 915/1834] 10.222.0.64 () {60 vars in 1153 bytes} [Tue Aug 2 10:00:05 2022] POST /identity/v3/projects => generated 401 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:00:05.963614 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c281100-ac50-4102-afe3-0bdabefa16ce None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:05.976399 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c281100-ac50-4102-afe3-0bdabefa16ce None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:06.056529 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c85627-2eea-4266-94c4-09f8c74ba744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:06.057338 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c85627-2eea-4266-94c4-09f8c74ba744 admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:06.057828 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c85627-2eea-4266-94c4-09f8c74ba744 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:06.058103 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c85627-2eea-4266-94c4-09f8c74ba744 admin admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:06.058894 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c85627-2eea-4266-94c4-09f8c74ba744 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:06.061689 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c85627-2eea-4266-94c4-09f8c74ba744 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:06.073233 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:06.090247 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 920/1835] 10.222.0.64 () {60 vars in 1144 bytes} [Tue Aug 2 10:00:05 2022] POST /identity/v3/users => generated 307 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:00:06.092056 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e088ff0-f2bb-4be8-a32e-fbf06d0079a8 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:06.103135 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e088ff0-f2bb-4be8-a32e-fbf06d0079a8 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:06.108639 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bd2029-4cde-43a7-bfeb-def97aa75d03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:06.109259 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bd2029-4cde-43a7-bfeb-def97aa75d03 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:06.109563 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bd2029-4cde-43a7-bfeb-def97aa75d03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:06.109825 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bd2029-4cde-43a7-bfeb-def97aa75d03 admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:06.110560 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bd2029-4cde-43a7-bfeb-def97aa75d03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:06.112163 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bd2029-4cde-43a7-bfeb-def97aa75d03 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:06.126434 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 916/1836] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:00:06 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:00:06.129605 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c85627-2eea-4266-94c4-09f8c74ba744 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:06.141661 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c85627-2eea-4266-94c4-09f8c74ba744 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:06.149625 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3734f8-d7ff-4951-8a87-39a202783abb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:06.150425 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3734f8-d7ff-4951-8a87-39a202783abb admin admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:06.150818 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3734f8-d7ff-4951-8a87-39a202783abb admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:06.151180 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3734f8-d7ff-4951-8a87-39a202783abb admin admin] PATH_INFO: `/v3/projects/b3cf5aae1b954291aa86eb75447dcc46/users/3e7372f0bc224c869c91a0bc3325db83/roles/1000cf318fa14915b8091599eaead5de` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:06.179270 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3734f8-d7ff-4951-8a87-39a202783abb admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3cf5aae1b954291aa86eb75447dcc46, user_id=3e7372f0bc224c869c91a0bc3325db83, role_id=1000cf318fa14915b8091599eaead5de)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:06.182113 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3734f8-d7ff-4951-8a87-39a202783abb admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:06.196835 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 921/1837] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 10:00:06 2022] PUT /identity/v3/projects/b3cf5aae1b954291aa86eb75447dcc46/users/3e7372f0bc224c869c91a0bc3325db83/roles/1000cf318fa14915b8091599eaead5de => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:00:06.200689 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bd2029-4cde-43a7-bfeb-def97aa75d03 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:06.219865 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bd2029-4cde-43a7-bfeb-def97aa75d03 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:06.288156 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9ab042-1411-4eda-b245-14c9787f73ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:06.288844 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9ab042-1411-4eda-b245-14c9787f73ef admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:06.289138 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9ab042-1411-4eda-b245-14c9787f73ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:06.289565 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9ab042-1411-4eda-b245-14c9787f73ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:06.290289 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9ab042-1411-4eda-b245-14c9787f73ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:06.291888 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9ab042-1411-4eda-b245-14c9787f73ef admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:06.310370 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 917/1838] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:00:06 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:00:06.314490 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3734f8-d7ff-4951-8a87-39a202783abb admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:06.326170 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3734f8-d7ff-4951-8a87-39a202783abb admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:06.333384 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489783d6-2ee3-43f8-aad0-dc1eaad05f52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:06.334102 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489783d6-2ee3-43f8-aad0-dc1eaad05f52 admin admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:06.334395 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489783d6-2ee3-43f8-aad0-dc1eaad05f52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:06.334675 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489783d6-2ee3-43f8-aad0-dc1eaad05f52 admin admin] PATH_INFO: `/v3/projects/b3cf5aae1b954291aa86eb75447dcc46/users/3e7372f0bc224c869c91a0bc3325db83/roles/1000cf318fa14915b8091599eaead5de` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:06.336707 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489783d6-2ee3-43f8-aad0-dc1eaad05f52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3cf5aae1b954291aa86eb75447dcc46, user_id=3e7372f0bc224c869c91a0bc3325db83, role_id=1000cf318fa14915b8091599eaead5de)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:06.339535 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489783d6-2ee3-43f8-aad0-dc1eaad05f52 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:06.351819 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 922/1839] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 10:00:06 2022] PUT /identity/v3/projects/b3cf5aae1b954291aa86eb75447dcc46/users/3e7372f0bc224c869c91a0bc3325db83/roles/1000cf318fa14915b8091599eaead5de => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:00:06.362666 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5247597c-2693-4817-a220-6f4e742116e8 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:06.362962 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5247597c-2693-4817-a220-6f4e742116e8 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:06.363251 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5247597c-2693-4817-a220-6f4e742116e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:06.390661 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-5247597c-2693-4817-a220-6f4e742116e8 None None] MFA Rules not processed for user `3e7372f0bc224c869c91a0bc3325db83`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:00:06.391859 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-5247597c-2693-4817-a220-6f4e742116e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:06.472458 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 918/1840] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:00:06 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:00:06.486503 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489783d6-2ee3-43f8-aad0-dc1eaad05f52 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:06.488587 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-489783d6-2ee3-43f8-aad0-dc1eaad05f52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:06.554287 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489783d6-2ee3-43f8-aad0-dc1eaad05f52 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:06.557965 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf128bb-f528-430a-bdfd-1c94d4b134a2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d7de62b9ee4039bdc79627ac7fb499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:06.558584 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf128bb-f528-430a-bdfd-1c94d4b134a2 service cinder] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:06.558882 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf128bb-f528-430a-bdfd-1c94d4b134a2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:06.559151 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf128bb-f528-430a-bdfd-1c94d4b134a2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:06.568858 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf128bb-f528-430a-bdfd-1c94d4b134a2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:06.570313 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf128bb-f528-430a-bdfd-1c94d4b134a2 service cinder] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:06.581138 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 923/1841] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:00:06 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:00:06.671725 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5247597c-2693-4817-a220-6f4e742116e8 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:06.685887 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5247597c-2693-4817-a220-6f4e742116e8 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:06.690488 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce0beaa-0a94-4de4-a459-876454ca76c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d7de62b9ee4039bdc79627ac7fb499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:06.691231 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce0beaa-0a94-4de4-a459-876454ca76c5 service cinder] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:06.691622 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce0beaa-0a94-4de4-a459-876454ca76c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:06.692001 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce0beaa-0a94-4de4-a459-876454ca76c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:06.748052 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:06.766307 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce0beaa-0a94-4de4-a459-876454ca76c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:06.767749 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce0beaa-0a94-4de4-a459-876454ca76c5 service cinder] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:06.779582 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 919/1842] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:00:06 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:00:10.645267 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf128bb-f528-430a-bdfd-1c94d4b134a2 service cinder] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:10.654266 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf128bb-f528-430a-bdfd-1c94d4b134a2 service cinder] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:10.657865 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446e9b81-6c7f-475e-929c-fa2f06b49c8c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d7de62b9ee4039bdc79627ac7fb499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:10.658469 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446e9b81-6c7f-475e-929c-fa2f06b49c8c service cinder] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:10.658749 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446e9b81-6c7f-475e-929c-fa2f06b49c8c service cinder] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:10.659046 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446e9b81-6c7f-475e-929c-fa2f06b49c8c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:10.660832 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-446e9b81-6c7f-475e-929c-fa2f06b49c8c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:10.722798 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446e9b81-6c7f-475e-929c-fa2f06b49c8c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:10.724238 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446e9b81-6c7f-475e-929c-fa2f06b49c8c service cinder] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:10.734396 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 924/1843] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:00:10 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:00:16.773837 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=7 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:17.873888 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce0beaa-0a94-4de4-a459-876454ca76c5 service cinder] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:17.875986 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-dce0beaa-0a94-4de4-a459-876454ca76c5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:17.949504 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce0beaa-0a94-4de4-a459-876454ca76c5 service cinder] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:17.954293 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0461e14e-62b0-46d6-a777-a67844fe99d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:17.954961 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0461e14e-62b0-46d6-a777-a67844fe99d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:17.955247 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0461e14e-62b0-46d6-a777-a67844fe99d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:17.956221 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0461e14e-62b0-46d6-a777-a67844fe99d1 admin admin] PATH_INFO: `/v3/users/011a88cb815542e3ba499ac572f9553f` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:17.957449 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0461e14e-62b0-46d6-a777-a67844fe99d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=011a88cb815542e3ba499ac572f9553f)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:17.959403 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0461e14e-62b0-46d6-a777-a67844fe99d1 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:18.006666 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-0461e14e-62b0-46d6-a777-a67844fe99d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '011a88cb815542e3ba499ac572f9553f'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:18.013385 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-0461e14e-62b0-46d6-a777-a67844fe99d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '011a88cb815542e3ba499ac572f9553f'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:18.014069 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-0461e14e-62b0-46d6-a777-a67844fe99d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '011a88cb815542e3ba499ac572f9553f'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:18.021076 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-0461e14e-62b0-46d6-a777-a67844fe99d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '011a88cb815542e3ba499ac572f9553f'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:18.034387 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-0461e14e-62b0-46d6-a777-a67844fe99d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '011a88cb815542e3ba499ac572f9553f'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:18.040146 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 920/1844] 10.222.0.64 () {58 vars in 1224 bytes} [Tue Aug 2 10:00:17 2022] DELETE /identity/v3/users/011a88cb815542e3ba499ac572f9553f => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:00:18.044829 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e822295-cd65-4807-b052-3942682c091f None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:18.045330 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e822295-cd65-4807-b052-3942682c091f None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:18.045431 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e822295-cd65-4807-b052-3942682c091f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:18.065130 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-5e822295-cd65-4807-b052-3942682c091f None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:00:18.066333 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-5e822295-cd65-4807-b052-3942682c091f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:18.140701 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:18.159907 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 925/1845] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:00:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:00:18.170167 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0461e14e-62b0-46d6-a777-a67844fe99d1 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:18.172080 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-0461e14e-62b0-46d6-a777-a67844fe99d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:18.263262 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0461e14e-62b0-46d6-a777-a67844fe99d1 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:18.267341 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b52969-7ef1-478b-a757-1a1b7054fc0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:18.268294 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b52969-7ef1-478b-a757-1a1b7054fc0c service neutron] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:18.268489 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b52969-7ef1-478b-a757-1a1b7054fc0c service neutron] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:18.268772 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b52969-7ef1-478b-a757-1a1b7054fc0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:18.279791 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b52969-7ef1-478b-a757-1a1b7054fc0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:18.282037 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b52969-7ef1-478b-a757-1a1b7054fc0c service neutron] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:18.293980 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 921/1846] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:00:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:00:18.467994 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e822295-cd65-4807-b052-3942682c091f None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:18.469661 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-5e822295-cd65-4807-b052-3942682c091f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:18.486711 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e822295-cd65-4807-b052-3942682c091f None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:18.509768 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62f2719-d3bf-4a61-88d5-0de358daf0a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:18.510764 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62f2719-d3bf-4a61-88d5-0de358daf0a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:18.511215 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62f2719-d3bf-4a61-88d5-0de358daf0a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:18.511753 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62f2719-d3bf-4a61-88d5-0de358daf0a5 admin admin] PATH_INFO: `/v3/users/39deafc81f694dd795de14b07e56b47b` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:18.514483 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62f2719-d3bf-4a61-88d5-0de358daf0a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=39deafc81f694dd795de14b07e56b47b)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:18.520383 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62f2719-d3bf-4a61-88d5-0de358daf0a5 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:18.604621 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-d62f2719-d3bf-4a61-88d5-0de358daf0a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39deafc81f694dd795de14b07e56b47b'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:18.613598 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-d62f2719-d3bf-4a61-88d5-0de358daf0a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39deafc81f694dd795de14b07e56b47b'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:18.622503 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-d62f2719-d3bf-4a61-88d5-0de358daf0a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39deafc81f694dd795de14b07e56b47b'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:18.623707 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-d62f2719-d3bf-4a61-88d5-0de358daf0a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39deafc81f694dd795de14b07e56b47b'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:18.624944 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-d62f2719-d3bf-4a61-88d5-0de358daf0a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39deafc81f694dd795de14b07e56b47b'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:18.667800 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 926/1847] 10.222.0.64 () {58 vars in 1224 bytes} [Tue Aug 2 10:00:18 2022] DELETE /identity/v3/users/39deafc81f694dd795de14b07e56b47b => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:00:18.672814 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fc1707-d121-4bab-b87a-22e55dfe2b0f None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:18.673588 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fc1707-d121-4bab-b87a-22e55dfe2b0f None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:18.673927 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fc1707-d121-4bab-b87a-22e55dfe2b0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:18.707529 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-54fc1707-d121-4bab-b87a-22e55dfe2b0f None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:00:18.710212 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-54fc1707-d121-4bab-b87a-22e55dfe2b0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:18.744981 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:18.793652 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62f2719-d3bf-4a61-88d5-0de358daf0a5 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:18.795452 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-d62f2719-d3bf-4a61-88d5-0de358daf0a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:18.813524 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 922/1848] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:00:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:00:18.828916 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fc1707-d121-4bab-b87a-22e55dfe2b0f None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:18.831591 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-54fc1707-d121-4bab-b87a-22e55dfe2b0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:18.897835 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fc1707-d121-4bab-b87a-22e55dfe2b0f None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:18.901571 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f3b696-056b-4f02-8715-793879e6091b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:18.902295 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f3b696-056b-4f02-8715-793879e6091b service neutron] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:18.902575 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f3b696-056b-4f02-8715-793879e6091b service neutron] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:18.902848 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f3b696-056b-4f02-8715-793879e6091b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:18.912163 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f3b696-056b-4f02-8715-793879e6091b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:18.913613 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f3b696-056b-4f02-8715-793879e6091b service neutron] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:18.916112 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62f2719-d3bf-4a61-88d5-0de358daf0a5 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:18.921177 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d3bc61-cb38-4894-81b5-a2e6522b4d3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:18.921938 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d3bc61-cb38-4894-81b5-a2e6522b4d3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:18.922275 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d3bc61-cb38-4894-81b5-a2e6522b4d3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:18.922558 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d3bc61-cb38-4894-81b5-a2e6522b4d3e admin admin] PATH_INFO: `/v3/projects/7c802bb85c414be9ad026f527e5b2ffc` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:18.923125 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 923/1849] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:00:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:00:18.923974 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d3bc61-cb38-4894-81b5-a2e6522b4d3e admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c802bb85c414be9ad026f527e5b2ffc)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:18.925989 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d3bc61-cb38-4894-81b5-a2e6522b4d3e admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:18.976570 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-62d3bc61-cb38-4894-81b5-a2e6522b4d3e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c802bb85c414be9ad026f527e5b2ffc', 'request_id': 'req-62d3bc61-cb38-4894-81b5-a2e6522b4d3e'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:18.984897 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-62d3bc61-cb38-4894-81b5-a2e6522b4d3e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c802bb85c414be9ad026f527e5b2ffc', 'request_id': 'req-62d3bc61-cb38-4894-81b5-a2e6522b4d3e'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:18.994014 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-62d3bc61-cb38-4894-81b5-a2e6522b4d3e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c802bb85c414be9ad026f527e5b2ffc', 'request_id': 'req-62d3bc61-cb38-4894-81b5-a2e6522b4d3e'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:19.006553 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:19.012798 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-62d3bc61-cb38-4894-81b5-a2e6522b4d3e admin admin] The token cache is being invalidate because project 7c802bb85c414be9ad026f527e5b2ffc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80153) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:00:19.013352 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-62d3bc61-cb38-4894-81b5-a2e6522b4d3e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:19.014856 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-62d3bc61-cb38-4894-81b5-a2e6522b4d3e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:19.019599 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 927/1850] 10.222.0.64 () {58 vars in 1233 bytes} [Tue Aug 2 10:00:18 2022] DELETE /identity/v3/projects/7c802bb85c414be9ad026f527e5b2ffc => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:00:19.023928 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f3b696-056b-4f02-8715-793879e6091b service neutron] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:19.035698 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f3b696-056b-4f02-8715-793879e6091b service neutron] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:19.122962 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd99e193-4ee6-4e86-b434-40288c644289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:19.123629 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd99e193-4ee6-4e86-b434-40288c644289 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:19.123915 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd99e193-4ee6-4e86-b434-40288c644289 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:19.124192 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd99e193-4ee6-4e86-b434-40288c644289 admin admin] PATH_INFO: `/v3/users/97dbad29b78645f88895f22f7039fbc9` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:19.125963 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd99e193-4ee6-4e86-b434-40288c644289 admin admin] RBAC: Authorizing `identity:delete_user(user_id=97dbad29b78645f88895f22f7039fbc9)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:19.128691 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd99e193-4ee6-4e86-b434-40288c644289 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:19.201738 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-dd99e193-4ee6-4e86-b434-40288c644289 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97dbad29b78645f88895f22f7039fbc9'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:19.213929 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-dd99e193-4ee6-4e86-b434-40288c644289 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97dbad29b78645f88895f22f7039fbc9'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:19.218408 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-dd99e193-4ee6-4e86-b434-40288c644289 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97dbad29b78645f88895f22f7039fbc9'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:19.232383 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-dd99e193-4ee6-4e86-b434-40288c644289 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97dbad29b78645f88895f22f7039fbc9'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:19.244199 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:19.256544 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-dd99e193-4ee6-4e86-b434-40288c644289 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97dbad29b78645f88895f22f7039fbc9'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:19.269677 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 924/1851] 10.222.0.64 () {58 vars in 1224 bytes} [Tue Aug 2 10:00:19 2022] DELETE /identity/v3/users/97dbad29b78645f88895f22f7039fbc9 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:00:19.311937 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d3bc61-cb38-4894-81b5-a2e6522b4d3e admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:19.314947 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-62d3bc61-cb38-4894-81b5-a2e6522b4d3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:19.330071 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d3bc61-cb38-4894-81b5-a2e6522b4d3e admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:19.402197 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cede610-ad19-45f5-9dd0-01b81f42457a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:19.402883 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cede610-ad19-45f5-9dd0-01b81f42457a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:19.403180 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cede610-ad19-45f5-9dd0-01b81f42457a admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:19.403454 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cede610-ad19-45f5-9dd0-01b81f42457a admin admin] PATH_INFO: `/v3/projects/ff68847a6af247d8a372053aa84500f2` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:19.404679 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cede610-ad19-45f5-9dd0-01b81f42457a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff68847a6af247d8a372053aa84500f2)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:19.406605 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cede610-ad19-45f5-9dd0-01b81f42457a admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:19.468580 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-9cede610-ad19-45f5-9dd0-01b81f42457a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff68847a6af247d8a372053aa84500f2', 'request_id': 'req-9cede610-ad19-45f5-9dd0-01b81f42457a'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:19.474831 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-9cede610-ad19-45f5-9dd0-01b81f42457a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff68847a6af247d8a372053aa84500f2', 'request_id': 'req-9cede610-ad19-45f5-9dd0-01b81f42457a'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:19.481517 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-9cede610-ad19-45f5-9dd0-01b81f42457a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff68847a6af247d8a372053aa84500f2', 'request_id': 'req-9cede610-ad19-45f5-9dd0-01b81f42457a'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:19.496119 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-9cede610-ad19-45f5-9dd0-01b81f42457a admin admin] The token cache is being invalidate because project ff68847a6af247d8a372053aa84500f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80153) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:00:19.496465 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-9cede610-ad19-45f5-9dd0-01b81f42457a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:19.497142 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-9cede610-ad19-45f5-9dd0-01b81f42457a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:19.499046 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 928/1852] 10.222.0.64 () {58 vars in 1233 bytes} [Tue Aug 2 10:00:19 2022] DELETE /identity/v3/projects/ff68847a6af247d8a372053aa84500f2 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:00:19.502957 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd99e193-4ee6-4e86-b434-40288c644289 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:19.504701 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-dd99e193-4ee6-4e86-b434-40288c644289 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:19.526758 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd99e193-4ee6-4e86-b434-40288c644289 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:19.579586 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cede610-ad19-45f5-9dd0-01b81f42457a admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:19.581195 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-9cede610-ad19-45f5-9dd0-01b81f42457a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:19.616825 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121c328e-84d5-4486-813e-404ee60f328d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:19.617616 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121c328e-84d5-4486-813e-404ee60f328d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:19.618066 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121c328e-84d5-4486-813e-404ee60f328d admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:19.618485 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121c328e-84d5-4486-813e-404ee60f328d admin admin] PATH_INFO: `/v3/users/3e7372f0bc224c869c91a0bc3325db83` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:19.619816 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121c328e-84d5-4486-813e-404ee60f328d admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e7372f0bc224c869c91a0bc3325db83)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:19.622521 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121c328e-84d5-4486-813e-404ee60f328d admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:19.671179 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cede610-ad19-45f5-9dd0-01b81f42457a admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:19.677441 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3425058a-1cf8-4313-a513-3b0a23ad9d14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:19.678099 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3425058a-1cf8-4313-a513-3b0a23ad9d14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:19.678456 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3425058a-1cf8-4313-a513-3b0a23ad9d14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:19.678769 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3425058a-1cf8-4313-a513-3b0a23ad9d14 admin admin] PATH_INFO: `/v3/projects/1a0bd282d8de482099bc32854206d3c6` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:19.679974 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3425058a-1cf8-4313-a513-3b0a23ad9d14 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a0bd282d8de482099bc32854206d3c6)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:19.681973 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3425058a-1cf8-4313-a513-3b0a23ad9d14 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:19.694444 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-121c328e-84d5-4486-813e-404ee60f328d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e7372f0bc224c869c91a0bc3325db83'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:19.701828 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-121c328e-84d5-4486-813e-404ee60f328d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e7372f0bc224c869c91a0bc3325db83'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:19.702838 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-121c328e-84d5-4486-813e-404ee60f328d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e7372f0bc224c869c91a0bc3325db83'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:19.710849 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-121c328e-84d5-4486-813e-404ee60f328d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e7372f0bc224c869c91a0bc3325db83'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:19.718743 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:19.725056 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-121c328e-84d5-4486-813e-404ee60f328d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e7372f0bc224c869c91a0bc3325db83'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:19.731996 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 925/1853] 10.222.0.64 () {58 vars in 1224 bytes} [Tue Aug 2 10:00:19 2022] DELETE /identity/v3/users/3e7372f0bc224c869c91a0bc3325db83 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:00:19.735168 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-3425058a-1cf8-4313-a513-3b0a23ad9d14 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a0bd282d8de482099bc32854206d3c6', 'request_id': 'req-3425058a-1cf8-4313-a513-3b0a23ad9d14'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:19.741493 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-3425058a-1cf8-4313-a513-3b0a23ad9d14 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a0bd282d8de482099bc32854206d3c6', 'request_id': 'req-3425058a-1cf8-4313-a513-3b0a23ad9d14'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:19.747922 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-3425058a-1cf8-4313-a513-3b0a23ad9d14 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a0bd282d8de482099bc32854206d3c6', 'request_id': 'req-3425058a-1cf8-4313-a513-3b0a23ad9d14'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:19.764427 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-3425058a-1cf8-4313-a513-3b0a23ad9d14 admin admin] The token cache is being invalidate because project 1a0bd282d8de482099bc32854206d3c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80153) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:00:19.765087 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-3425058a-1cf8-4313-a513-3b0a23ad9d14 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:19.765868 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-3425058a-1cf8-4313-a513-3b0a23ad9d14 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:19.768322 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 929/1854] 10.222.0.64 () {58 vars in 1233 bytes} [Tue Aug 2 10:00:19 2022] DELETE /identity/v3/projects/1a0bd282d8de482099bc32854206d3c6 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:00:19.773229 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80171c82-fc5b-4472-9aa4-914e4a9c9633 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:19.773687 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80171c82-fc5b-4472-9aa4-914e4a9c9633 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:19.774257 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80171c82-fc5b-4472-9aa4-914e4a9c9633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:19.798182 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-80171c82-fc5b-4472-9aa4-914e4a9c9633 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:00:19.800328 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-80171c82-fc5b-4472-9aa4-914e4a9c9633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:19.873145 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 926/1855] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:00:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:00:19.882156 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06988de-5178-4148-a235-d806a0daa7e8 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:19.882452 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06988de-5178-4148-a235-d806a0daa7e8 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:19.882736 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06988de-5178-4148-a235-d806a0daa7e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:19.909622 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-a06988de-5178-4148-a235-d806a0daa7e8 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:00:19.910896 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-a06988de-5178-4148-a235-d806a0daa7e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:19.918890 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 930/1856] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:00:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:00:19.923201 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80171c82-fc5b-4472-9aa4-914e4a9c9633 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:19.929434 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:19.933510 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80171c82-fc5b-4472-9aa4-914e4a9c9633 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:19.939198 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51171476-d8a7-4487-9be6-9b977a8a7c4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:19.939979 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51171476-d8a7-4487-9be6-9b977a8a7c4b admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:19.941051 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51171476-d8a7-4487-9be6-9b977a8a7c4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:19.941597 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51171476-d8a7-4487-9be6-9b977a8a7c4b admin admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:19.942423 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51171476-d8a7-4487-9be6-9b977a8a7c4b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:00:19.942836 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51171476-d8a7-4487-9be6-9b977a8a7c4b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:19.944817 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51171476-d8a7-4487-9be6-9b977a8a7c4b admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:19.961059 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 927/1857] 10.222.0.64 () {58 vars in 1153 bytes} [Tue Aug 2 10:00:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:19.962209 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06988de-5178-4148-a235-d806a0daa7e8 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:19.972636 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06988de-5178-4148-a235-d806a0daa7e8 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:19.977539 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fdae9f-ed05-4f80-acb2-1fd90236c005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:19.978240 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fdae9f-ed05-4f80-acb2-1fd90236c005 admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:19.978606 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fdae9f-ed05-4f80-acb2-1fd90236c005 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:19.978950 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fdae9f-ed05-4f80-acb2-1fd90236c005 admin admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:19.979699 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fdae9f-ed05-4f80-acb2-1fd90236c005 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:19.981808 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fdae9f-ed05-4f80-acb2-1fd90236c005 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:20.011487 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 931/1858] 10.222.0.64 () {60 vars in 1153 bytes} [Tue Aug 2 10:00:19 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:00:20.015434 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51171476-d8a7-4487-9be6-9b977a8a7c4b admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:20.025926 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51171476-d8a7-4487-9be6-9b977a8a7c4b admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:20.085093 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f18ce1f-4ed1-4750-9e19-474889f409f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:20.085904 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f18ce1f-4ed1-4750-9e19-474889f409f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:20.086188 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f18ce1f-4ed1-4750-9e19-474889f409f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:20.086461 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f18ce1f-4ed1-4750-9e19-474889f409f7 admin admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:20.087147 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f18ce1f-4ed1-4750-9e19-474889f409f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:20.089122 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f18ce1f-4ed1-4750-9e19-474889f409f7 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:20.112847 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 928/1859] 10.222.0.64 () {60 vars in 1144 bytes} [Tue Aug 2 10:00:20 2022] POST /identity/v3/users => generated 292 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:00:20.117416 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fdae9f-ed05-4f80-acb2-1fd90236c005 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:20.128165 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fdae9f-ed05-4f80-acb2-1fd90236c005 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:20.133078 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e4a1ad-6b32-4ac8-aa3f-4ff4cfa34441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:20.133721 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e4a1ad-6b32-4ac8-aa3f-4ff4cfa34441 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:20.134010 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e4a1ad-6b32-4ac8-aa3f-4ff4cfa34441 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:20.134289 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e4a1ad-6b32-4ac8-aa3f-4ff4cfa34441 admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:20.135028 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e4a1ad-6b32-4ac8-aa3f-4ff4cfa34441 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:20.136706 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e4a1ad-6b32-4ac8-aa3f-4ff4cfa34441 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:20.148218 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 932/1860] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:00:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:00:20.152577 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f18ce1f-4ed1-4750-9e19-474889f409f7 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:20.165409 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f18ce1f-4ed1-4750-9e19-474889f409f7 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:20.170306 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f425f9-719b-4f6c-9453-d9f2bfe4c4d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:20.171011 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f425f9-719b-4f6c-9453-d9f2bfe4c4d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:20.171371 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f425f9-719b-4f6c-9453-d9f2bfe4c4d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:20.171814 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f425f9-719b-4f6c-9453-d9f2bfe4c4d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:20.172568 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f425f9-719b-4f6c-9453-d9f2bfe4c4d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:20.174198 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f425f9-719b-4f6c-9453-d9f2bfe4c4d5 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:20.189546 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 929/1861] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:00:20 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:00:20.191282 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e4a1ad-6b32-4ac8-aa3f-4ff4cfa34441 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:20.202439 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e4a1ad-6b32-4ac8-aa3f-4ff4cfa34441 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:20.208404 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5098d3d2-36a2-4073-8d75-8dcd117c5543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:20.209216 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5098d3d2-36a2-4073-8d75-8dcd117c5543 admin admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:20.209599 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5098d3d2-36a2-4073-8d75-8dcd117c5543 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:20.209974 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5098d3d2-36a2-4073-8d75-8dcd117c5543 admin admin] PATH_INFO: `/v3/projects/23043e102aa243049eaee918b0bb89c3/users/937c6b9981664855bf528ff99dab5a8a/roles/66c1074780954f8e87ed11a5c4e8c36a` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:20.231214 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5098d3d2-36a2-4073-8d75-8dcd117c5543 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23043e102aa243049eaee918b0bb89c3, user_id=937c6b9981664855bf528ff99dab5a8a, role_id=66c1074780954f8e87ed11a5c4e8c36a)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:20.233996 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5098d3d2-36a2-4073-8d75-8dcd117c5543 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:20.248860 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 933/1862] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 10:00:20 2022] PUT /identity/v3/projects/23043e102aa243049eaee918b0bb89c3/users/937c6b9981664855bf528ff99dab5a8a/roles/66c1074780954f8e87ed11a5c4e8c36a => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:00:20.259908 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee25049b-110c-48b1-962d-731bf5afeabf None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:20.260697 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee25049b-110c-48b1-962d-731bf5afeabf None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:20.260697 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee25049b-110c-48b1-962d-731bf5afeabf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:20.285049 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5098d3d2-36a2-4073-8d75-8dcd117c5543 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:20.286816 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-5098d3d2-36a2-4073-8d75-8dcd117c5543 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:20.307155 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-ee25049b-110c-48b1-962d-731bf5afeabf None None] MFA Rules not processed for user `937c6b9981664855bf528ff99dab5a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:00:20.308465 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-ee25049b-110c-48b1-962d-731bf5afeabf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:20.361988 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5098d3d2-36a2-4073-8d75-8dcd117c5543 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:20.366844 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d387251-5cb4-48a5-9455-36f4ffeed73b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:20.367492 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d387251-5cb4-48a5-9455-36f4ffeed73b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:20.367775 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d387251-5cb4-48a5-9455-36f4ffeed73b admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:20.368052 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d387251-5cb4-48a5-9455-36f4ffeed73b admin admin] PATH_INFO: `/v3/projects/b3cf5aae1b954291aa86eb75447dcc46` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:20.369248 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d387251-5cb4-48a5-9455-36f4ffeed73b admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3cf5aae1b954291aa86eb75447dcc46)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:20.371295 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d387251-5cb4-48a5-9455-36f4ffeed73b admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:20.388443 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 930/1863] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:00:20 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:00:20.392642 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee25049b-110c-48b1-962d-731bf5afeabf None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:20.405430 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee25049b-110c-48b1-962d-731bf5afeabf None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:20.411452 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfa666a-aa86-4769-a136-05f16692b836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:20.412391 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfa666a-aa86-4769-a136-05f16692b836 admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:20.412774 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfa666a-aa86-4769-a136-05f16692b836 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:20.413093 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfa666a-aa86-4769-a136-05f16692b836 admin admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:20.424407 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfa666a-aa86-4769-a136-05f16692b836 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:20.424407 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfa666a-aa86-4769-a136-05f16692b836 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:20.428353 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-8d387251-5cb4-48a5-9455-36f4ffeed73b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3cf5aae1b954291aa86eb75447dcc46', 'request_id': 'req-8d387251-5cb4-48a5-9455-36f4ffeed73b'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:20.433623 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:20.434604 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-8d387251-5cb4-48a5-9455-36f4ffeed73b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3cf5aae1b954291aa86eb75447dcc46', 'request_id': 'req-8d387251-5cb4-48a5-9455-36f4ffeed73b'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:20.444951 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-8d387251-5cb4-48a5-9455-36f4ffeed73b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3cf5aae1b954291aa86eb75447dcc46', 'request_id': 'req-8d387251-5cb4-48a5-9455-36f4ffeed73b'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:20.460288 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 931/1864] 10.222.0.64 () {60 vars in 1153 bytes} [Tue Aug 2 10:00:20 2022] POST /identity/v3/projects => generated 383 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:00:20.460558 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-8d387251-5cb4-48a5-9455-36f4ffeed73b admin admin] The token cache is being invalidate because project b3cf5aae1b954291aa86eb75447dcc46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80153) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:00:20.460851 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-8d387251-5cb4-48a5-9455-36f4ffeed73b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:20.461501 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-8d387251-5cb4-48a5-9455-36f4ffeed73b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:20.463058 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 934/1865] 10.222.0.64 () {58 vars in 1233 bytes} [Tue Aug 2 10:00:20 2022] DELETE /identity/v3/projects/b3cf5aae1b954291aa86eb75447dcc46 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:00:20.463935 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfa666a-aa86-4769-a136-05f16692b836 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:20.465536 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-9bfa666a-aa86-4769-a136-05f16692b836 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:20.469312 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dbbd5d-c7db-41b3-82a9-cf4f5f0f6572 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:20.469673 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dbbd5d-c7db-41b3-82a9-cf4f5f0f6572 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:20.470009 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dbbd5d-c7db-41b3-82a9-cf4f5f0f6572 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:20.492455 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-89dbbd5d-c7db-41b3-82a9-cf4f5f0f6572 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:00:20.493735 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-89dbbd5d-c7db-41b3-82a9-cf4f5f0f6572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:20.516241 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:20.541055 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfa666a-aa86-4769-a136-05f16692b836 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:20.545849 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd297ff-3b4c-46d7-ba8d-f7fa8bc4f3eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:20.546487 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd297ff-3b4c-46d7-ba8d-f7fa8bc4f3eb admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:20.546774 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd297ff-3b4c-46d7-ba8d-f7fa8bc4f3eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:20.547051 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd297ff-3b4c-46d7-ba8d-f7fa8bc4f3eb admin admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:20.547759 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd297ff-3b4c-46d7-ba8d-f7fa8bc4f3eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:20.550312 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd297ff-3b4c-46d7-ba8d-f7fa8bc4f3eb admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:20.572299 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 935/1866] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:00:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:00:20.576805 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c476403-4c4e-4379-a2d0-10df03755a01 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:20.577104 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c476403-4c4e-4379-a2d0-10df03755a01 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:20.577387 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c476403-4c4e-4379-a2d0-10df03755a01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:20.581941 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 932/1867] 10.222.0.64 () {60 vars in 1144 bytes} [Tue Aug 2 10:00:20 2022] POST /identity/v3/users => generated 298 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:00:20.582766 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd297ff-3b4c-46d7-ba8d-f7fa8bc4f3eb admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:20.592386 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd297ff-3b4c-46d7-ba8d-f7fa8bc4f3eb admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:20.597125 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7ef6cd-c943-4039-a0f6-873ffee320eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:20.597362 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-2c476403-4c4e-4379-a2d0-10df03755a01 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:00:20.597789 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7ef6cd-c943-4039-a0f6-873ffee320eb admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:20.598077 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7ef6cd-c943-4039-a0f6-873ffee320eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:20.598357 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7ef6cd-c943-4039-a0f6-873ffee320eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:20.598490 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-2c476403-4c4e-4379-a2d0-10df03755a01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:20.599064 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7ef6cd-c943-4039-a0f6-873ffee320eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:20.600698 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7ef6cd-c943-4039-a0f6-873ffee320eb admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:20.606781 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 936/1868] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:00:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:00:20.610958 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c476403-4c4e-4379-a2d0-10df03755a01 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:20.615773 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 933/1869] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:00:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:00:20.616844 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7ef6cd-c943-4039-a0f6-873ffee320eb admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:20.621086 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c476403-4c4e-4379-a2d0-10df03755a01 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:20.625895 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176aba32-7aab-4071-9cee-de0a4cdf955d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:20.626405 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7ef6cd-c943-4039-a0f6-873ffee320eb admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:20.626526 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176aba32-7aab-4071-9cee-de0a4cdf955d admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:20.626767 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176aba32-7aab-4071-9cee-de0a4cdf955d admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:20.627052 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176aba32-7aab-4071-9cee-de0a4cdf955d admin admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:20.628366 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176aba32-7aab-4071-9cee-de0a4cdf955d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:00:20.628678 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176aba32-7aab-4071-9cee-de0a4cdf955d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:20.630284 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176aba32-7aab-4071-9cee-de0a4cdf955d admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:20.631732 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd3bdfb-2d48-480c-964e-90dbb1f51f23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:20.632416 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd3bdfb-2d48-480c-964e-90dbb1f51f23 admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:20.632715 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd3bdfb-2d48-480c-964e-90dbb1f51f23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:20.633026 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd3bdfb-2d48-480c-964e-90dbb1f51f23 admin admin] PATH_INFO: `/v3/projects/68caab932ca04287a3da8319fe815adf/users/c0026497d6e8406d8079296cb990d524/roles/1000cf318fa14915b8091599eaead5de` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:20.642180 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 937/1870] 10.222.0.64 () {58 vars in 1153 bytes} [Tue Aug 2 10:00:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:00:20.646475 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176aba32-7aab-4071-9cee-de0a4cdf955d admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:20.647914 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd3bdfb-2d48-480c-964e-90dbb1f51f23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=68caab932ca04287a3da8319fe815adf, user_id=c0026497d6e8406d8079296cb990d524, role_id=1000cf318fa14915b8091599eaead5de)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:20.650594 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd3bdfb-2d48-480c-964e-90dbb1f51f23 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:20.656493 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176aba32-7aab-4071-9cee-de0a4cdf955d admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:20.661765 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3c0304-c39c-4e7d-81df-29a1f6de70e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:20.662360 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3c0304-c39c-4e7d-81df-29a1f6de70e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:20.662648 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3c0304-c39c-4e7d-81df-29a1f6de70e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:20.662919 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3c0304-c39c-4e7d-81df-29a1f6de70e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:20.663581 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3c0304-c39c-4e7d-81df-29a1f6de70e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:20.664941 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 934/1871] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 10:00:20 2022] PUT /identity/v3/projects/68caab932ca04287a3da8319fe815adf/users/c0026497d6e8406d8079296cb990d524/roles/1000cf318fa14915b8091599eaead5de => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:00:20.668002 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3c0304-c39c-4e7d-81df-29a1f6de70e9 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:20.668635 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd3bdfb-2d48-480c-964e-90dbb1f51f23 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:20.697735 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 938/1872] 10.222.0.64 () {60 vars in 1153 bytes} [Tue Aug 2 10:00:20 2022] POST /identity/v3/projects => generated 375 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:00:20.701631 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3c0304-c39c-4e7d-81df-29a1f6de70e9 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:20.739881 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd3bdfb-2d48-480c-964e-90dbb1f51f23 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:20.768370 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3c0304-c39c-4e7d-81df-29a1f6de70e9 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:20.773330 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3d5dee-9c77-42eb-8735-e93a40b8ad5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:20.774155 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3d5dee-9c77-42eb-8735-e93a40b8ad5e admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:20.774541 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3d5dee-9c77-42eb-8735-e93a40b8ad5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:20.774824 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3d5dee-9c77-42eb-8735-e93a40b8ad5e admin admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:20.775495 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3d5dee-9c77-42eb-8735-e93a40b8ad5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:20.777626 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3d5dee-9c77-42eb-8735-e93a40b8ad5e admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:20.800975 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 939/1873] 10.222.0.64 () {60 vars in 1144 bytes} [Tue Aug 2 10:00:20 2022] POST /identity/v3/users => generated 288 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:00:20.804915 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3d5dee-9c77-42eb-8735-e93a40b8ad5e admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:20.805050 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b23df8-ecee-4312-8405-8fe01d7cccc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:20.805874 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b23df8-ecee-4312-8405-8fe01d7cccc5 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:20.806113 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b23df8-ecee-4312-8405-8fe01d7cccc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:20.806385 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b23df8-ecee-4312-8405-8fe01d7cccc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:20.807108 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b23df8-ecee-4312-8405-8fe01d7cccc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:20.808740 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b23df8-ecee-4312-8405-8fe01d7cccc5 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:20.817287 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3d5dee-9c77-42eb-8735-e93a40b8ad5e admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:20.820448 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 935/1874] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:00:20 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:00:20.822442 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9dc8b3-2596-4c66-aaee-6b7ff1b3968c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:20.823327 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9dc8b3-2596-4c66-aaee-6b7ff1b3968c admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:20.823622 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9dc8b3-2596-4c66-aaee-6b7ff1b3968c admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:20.823901 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9dc8b3-2596-4c66-aaee-6b7ff1b3968c admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:20.824225 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b23df8-ecee-4312-8405-8fe01d7cccc5 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:20.824639 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9dc8b3-2596-4c66-aaee-6b7ff1b3968c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:20.826242 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9dc8b3-2596-4c66-aaee-6b7ff1b3968c admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:20.836882 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b23df8-ecee-4312-8405-8fe01d7cccc5 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:20.840626 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 940/1875] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:00:20 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:00:20.841246 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9dc8b3-2596-4c66-aaee-6b7ff1b3968c admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:20.845794 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a35b566-2f89-4871-984b-6d989c539c6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:20.847368 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a35b566-2f89-4871-984b-6d989c539c6f admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:20.848968 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a35b566-2f89-4871-984b-6d989c539c6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:20.849262 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a35b566-2f89-4871-984b-6d989c539c6f admin admin] PATH_INFO: `/v3/projects/68caab932ca04287a3da8319fe815adf/users/c0026497d6e8406d8079296cb990d524/roles/1000cf318fa14915b8091599eaead5de` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:20.851631 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a35b566-2f89-4871-984b-6d989c539c6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=68caab932ca04287a3da8319fe815adf, user_id=c0026497d6e8406d8079296cb990d524, role_id=1000cf318fa14915b8091599eaead5de)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:20.851922 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9dc8b3-2596-4c66-aaee-6b7ff1b3968c admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:20.854974 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a35b566-2f89-4871-984b-6d989c539c6f admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:20.865995 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee11a35f-906b-4f83-bb3b-495b4418cccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:20.866591 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 936/1876] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 10:00:20 2022] PUT /identity/v3/projects/68caab932ca04287a3da8319fe815adf/users/c0026497d6e8406d8079296cb990d524/roles/1000cf318fa14915b8091599eaead5de => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:00:20.867088 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee11a35f-906b-4f83-bb3b-495b4418cccc admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:20.867443 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee11a35f-906b-4f83-bb3b-495b4418cccc admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:20.867790 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee11a35f-906b-4f83-bb3b-495b4418cccc admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:20.868727 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee11a35f-906b-4f83-bb3b-495b4418cccc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:20.870428 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee11a35f-906b-4f83-bb3b-495b4418cccc admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:20.878763 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4252efb-62ef-4481-b4a2-6bab5ae7c367 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:20.879056 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4252efb-62ef-4481-b4a2-6bab5ae7c367 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:20.879338 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4252efb-62ef-4481-b4a2-6bab5ae7c367 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:20.884732 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 941/1877] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:00:20 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:00:20.889053 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee11a35f-906b-4f83-bb3b-495b4418cccc admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:20.899025 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee11a35f-906b-4f83-bb3b-495b4418cccc admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:20.907509 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:20.920467 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-d4252efb-62ef-4481-b4a2-6bab5ae7c367 None None] MFA Rules not processed for user `c0026497d6e8406d8079296cb990d524`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:00:20.921945 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-d4252efb-62ef-4481-b4a2-6bab5ae7c367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:20.931737 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:20.965616 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c990a5-b9bc-4a40-acd3-bbc92c806123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:20.966303 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c990a5-b9bc-4a40-acd3-bbc92c806123 admin admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:20.966591 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c990a5-b9bc-4a40-acd3-bbc92c806123 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:20.966870 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c990a5-b9bc-4a40-acd3-bbc92c806123 admin admin] PATH_INFO: `/v3/projects/f15edeeaae474d4ab94e9dcb652da720/users/de78efedd8ae4e3e8801261ba6e8e5fb/roles/66c1074780954f8e87ed11a5c4e8c36a` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:20.981788 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c990a5-b9bc-4a40-acd3-bbc92c806123 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f15edeeaae474d4ab94e9dcb652da720, user_id=de78efedd8ae4e3e8801261ba6e8e5fb, role_id=66c1074780954f8e87ed11a5c4e8c36a)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:20.984563 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c990a5-b9bc-4a40-acd3-bbc92c806123 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:20.994126 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 937/1878] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:00:20 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:00:20.999307 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 942/1879] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 10:00:20 2022] PUT /identity/v3/projects/f15edeeaae474d4ab94e9dcb652da720/users/de78efedd8ae4e3e8801261ba6e8e5fb/roles/66c1074780954f8e87ed11a5c4e8c36a => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:00:21.009261 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c990a5-b9bc-4a40-acd3-bbc92c806123 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:21.009931 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762277ef-c5c4-42d2-9412-82c9d518e2f3 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:21.010338 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762277ef-c5c4-42d2-9412-82c9d518e2f3 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:21.010704 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762277ef-c5c4-42d2-9412-82c9d518e2f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:21.011535 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-f0c990a5-b9bc-4a40-acd3-bbc92c806123 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:21.039162 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-762277ef-c5c4-42d2-9412-82c9d518e2f3 None None] MFA Rules not processed for user `de78efedd8ae4e3e8801261ba6e8e5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:00:21.040366 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-762277ef-c5c4-42d2-9412-82c9d518e2f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:21.078344 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c990a5-b9bc-4a40-acd3-bbc92c806123 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:21.082210 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28c7af3-0443-417b-81f2-c7e656eee408 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d7de62b9ee4039bdc79627ac7fb499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:21.082830 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28c7af3-0443-417b-81f2-c7e656eee408 service cinder] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:21.083117 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28c7af3-0443-417b-81f2-c7e656eee408 service cinder] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:21.083398 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28c7af3-0443-417b-81f2-c7e656eee408 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:21.114770 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 938/1880] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:00:21 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:00:21.125486 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762277ef-c5c4-42d2-9412-82c9d518e2f3 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:21.128278 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-762277ef-c5c4-42d2-9412-82c9d518e2f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:21.163786 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28c7af3-0443-417b-81f2-c7e656eee408 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:21.165278 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28c7af3-0443-417b-81f2-c7e656eee408 service cinder] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:21.176460 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 943/1881] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:00:21 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:00:21.224525 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762277ef-c5c4-42d2-9412-82c9d518e2f3 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:21.228487 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507633d6-0ced-407b-b365-07a302ee7dab service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04fb1dbe260a4721aaa0bd0493260d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:21.229080 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507633d6-0ced-407b-b365-07a302ee7dab service glance] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:21.229383 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507633d6-0ced-407b-b365-07a302ee7dab service glance] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:21.229653 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507633d6-0ced-407b-b365-07a302ee7dab service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:21.240426 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507633d6-0ced-407b-b365-07a302ee7dab service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:21.240426 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507633d6-0ced-407b-b365-07a302ee7dab service glance] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:21.249580 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 939/1882] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:00:21 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:00:21.250390 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28c7af3-0443-417b-81f2-c7e656eee408 service cinder] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:21.259137 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28c7af3-0443-417b-81f2-c7e656eee408 service cinder] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:21.263138 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd4b09b-fb23-4617-a1cd-4e69d6ce1e69 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d7de62b9ee4039bdc79627ac7fb499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:21.263784 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd4b09b-fb23-4617-a1cd-4e69d6ce1e69 service cinder] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:21.264071 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd4b09b-fb23-4617-a1cd-4e69d6ce1e69 service cinder] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:21.264394 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd4b09b-fb23-4617-a1cd-4e69d6ce1e69 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:21.266277 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-dfd4b09b-fb23-4617-a1cd-4e69d6ce1e69 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:21.314476 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:21.338351 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd4b09b-fb23-4617-a1cd-4e69d6ce1e69 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:21.339919 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd4b09b-fb23-4617-a1cd-4e69d6ce1e69 service cinder] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:21.353517 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 944/1883] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:00:21 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:00:21.662882 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507633d6-0ced-407b-b365-07a302ee7dab service glance] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:21.687524 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:21.744350 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507633d6-0ced-407b-b365-07a302ee7dab service glance] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:21.749416 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23866b30-b545-4ad4-8c83-deb29b3bfba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:21.750500 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23866b30-b545-4ad4-8c83-deb29b3bfba3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:21.750500 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23866b30-b545-4ad4-8c83-deb29b3bfba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:21.750878 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23866b30-b545-4ad4-8c83-deb29b3bfba3 admin admin] PATH_INFO: `/v3/users/937c6b9981664855bf528ff99dab5a8a` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:21.753211 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23866b30-b545-4ad4-8c83-deb29b3bfba3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=937c6b9981664855bf528ff99dab5a8a)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:21.755348 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23866b30-b545-4ad4-8c83-deb29b3bfba3 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:21.814035 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-23866b30-b545-4ad4-8c83-deb29b3bfba3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '937c6b9981664855bf528ff99dab5a8a'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:21.821836 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-23866b30-b545-4ad4-8c83-deb29b3bfba3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '937c6b9981664855bf528ff99dab5a8a'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:21.822549 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-23866b30-b545-4ad4-8c83-deb29b3bfba3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '937c6b9981664855bf528ff99dab5a8a'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:21.830318 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-23866b30-b545-4ad4-8c83-deb29b3bfba3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '937c6b9981664855bf528ff99dab5a8a'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:21.843030 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-23866b30-b545-4ad4-8c83-deb29b3bfba3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '937c6b9981664855bf528ff99dab5a8a'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:21.849114 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 940/1884] 10.222.0.64 () {58 vars in 1224 bytes} [Tue Aug 2 10:00:21 2022] DELETE /identity/v3/users/937c6b9981664855bf528ff99dab5a8a => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:00:21.853704 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443ab67a-41e3-41a0-961e-c905db4ed0ea None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:21.854142 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443ab67a-41e3-41a0-961e-c905db4ed0ea None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:21.854262 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443ab67a-41e3-41a0-961e-c905db4ed0ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:21.873849 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-443ab67a-41e3-41a0-961e-c905db4ed0ea None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:00:21.875040 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-443ab67a-41e3-41a0-961e-c905db4ed0ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:21.938627 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 945/1885] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:00:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:00:21.948849 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23866b30-b545-4ad4-8c83-deb29b3bfba3 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:21.950355 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-23866b30-b545-4ad4-8c83-deb29b3bfba3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:22.011689 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23866b30-b545-4ad4-8c83-deb29b3bfba3 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:22.015352 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d946b683-1dfd-43d4-932e-10da43f176ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:22.015983 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d946b683-1dfd-43d4-932e-10da43f176ef service neutron] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:22.016292 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d946b683-1dfd-43d4-932e-10da43f176ef service neutron] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:22.016576 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d946b683-1dfd-43d4-932e-10da43f176ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:22.026799 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d946b683-1dfd-43d4-932e-10da43f176ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:22.028267 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d946b683-1dfd-43d4-932e-10da43f176ef service neutron] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:22.037961 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 941/1886] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:00:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:00:22.354406 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443ab67a-41e3-41a0-961e-c905db4ed0ea None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:22.355882 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-443ab67a-41e3-41a0-961e-c905db4ed0ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:22.368781 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443ab67a-41e3-41a0-961e-c905db4ed0ea None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:22.374496 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d99cb4-3b8c-492c-bc28-6573783b0e25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:22.375691 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d99cb4-3b8c-492c-bc28-6573783b0e25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:22.376158 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d99cb4-3b8c-492c-bc28-6573783b0e25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:22.376674 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d99cb4-3b8c-492c-bc28-6573783b0e25 admin admin] PATH_INFO: `/v3/projects/23043e102aa243049eaee918b0bb89c3` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:22.378624 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d99cb4-3b8c-492c-bc28-6573783b0e25 admin admin] RBAC: Authorizing `identity:delete_project(project_id=23043e102aa243049eaee918b0bb89c3)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:22.380740 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d99cb4-3b8c-492c-bc28-6573783b0e25 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:22.444911 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-e6d99cb4-3b8c-492c-bc28-6573783b0e25 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23043e102aa243049eaee918b0bb89c3', 'request_id': 'req-e6d99cb4-3b8c-492c-bc28-6573783b0e25'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:22.450825 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-e6d99cb4-3b8c-492c-bc28-6573783b0e25 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23043e102aa243049eaee918b0bb89c3', 'request_id': 'req-e6d99cb4-3b8c-492c-bc28-6573783b0e25'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:22.457548 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-e6d99cb4-3b8c-492c-bc28-6573783b0e25 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23043e102aa243049eaee918b0bb89c3', 'request_id': 'req-e6d99cb4-3b8c-492c-bc28-6573783b0e25'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:22.470775 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-e6d99cb4-3b8c-492c-bc28-6573783b0e25 admin admin] The token cache is being invalidate because project 23043e102aa243049eaee918b0bb89c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80153) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:00:22.471064 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-e6d99cb4-3b8c-492c-bc28-6573783b0e25 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:22.472186 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-e6d99cb4-3b8c-492c-bc28-6573783b0e25 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:22.473792 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 946/1887] 10.222.0.64 () {58 vars in 1233 bytes} [Tue Aug 2 10:00:22 2022] DELETE /identity/v3/projects/23043e102aa243049eaee918b0bb89c3 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:00:22.478025 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d946b683-1dfd-43d4-932e-10da43f176ef service neutron] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:22.488489 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d946b683-1dfd-43d4-932e-10da43f176ef service neutron] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:22.536207 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:22.554474 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1952f3b-f664-4f2f-a643-5566aa21503a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:22.555115 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1952f3b-f664-4f2f-a643-5566aa21503a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:22.555423 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1952f3b-f664-4f2f-a643-5566aa21503a admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:22.555694 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1952f3b-f664-4f2f-a643-5566aa21503a admin admin] PATH_INFO: `/v3/users/c0026497d6e8406d8079296cb990d524` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:22.556887 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1952f3b-f664-4f2f-a643-5566aa21503a admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0026497d6e8406d8079296cb990d524)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:22.558857 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1952f3b-f664-4f2f-a643-5566aa21503a admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:22.605516 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-c1952f3b-f664-4f2f-a643-5566aa21503a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0026497d6e8406d8079296cb990d524'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:22.613201 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-c1952f3b-f664-4f2f-a643-5566aa21503a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0026497d6e8406d8079296cb990d524'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:22.613945 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-c1952f3b-f664-4f2f-a643-5566aa21503a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0026497d6e8406d8079296cb990d524'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:22.621878 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-c1952f3b-f664-4f2f-a643-5566aa21503a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0026497d6e8406d8079296cb990d524'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:22.634348 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-c1952f3b-f664-4f2f-a643-5566aa21503a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0026497d6e8406d8079296cb990d524'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:22.640464 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 942/1888] 10.222.0.64 () {58 vars in 1224 bytes} [Tue Aug 2 10:00:22 2022] DELETE /identity/v3/users/c0026497d6e8406d8079296cb990d524 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:00:22.964355 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d99cb4-3b8c-492c-bc28-6573783b0e25 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:22.965924 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-e6d99cb4-3b8c-492c-bc28-6573783b0e25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:22.981471 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d99cb4-3b8c-492c-bc28-6573783b0e25 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:23.042076 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af1cf10-d095-4ce0-8edd-7aacf04c698b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:23.042720 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af1cf10-d095-4ce0-8edd-7aacf04c698b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:23.043029 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af1cf10-d095-4ce0-8edd-7aacf04c698b admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:23.043311 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af1cf10-d095-4ce0-8edd-7aacf04c698b admin admin] PATH_INFO: `/v3/projects/68caab932ca04287a3da8319fe815adf` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:23.044498 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af1cf10-d095-4ce0-8edd-7aacf04c698b admin admin] RBAC: Authorizing `identity:delete_project(project_id=68caab932ca04287a3da8319fe815adf)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:23.046564 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af1cf10-d095-4ce0-8edd-7aacf04c698b admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:23.055466 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:23.099490 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-7af1cf10-d095-4ce0-8edd-7aacf04c698b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68caab932ca04287a3da8319fe815adf', 'request_id': 'req-7af1cf10-d095-4ce0-8edd-7aacf04c698b'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:23.105572 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-7af1cf10-d095-4ce0-8edd-7aacf04c698b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68caab932ca04287a3da8319fe815adf', 'request_id': 'req-7af1cf10-d095-4ce0-8edd-7aacf04c698b'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:23.112059 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-7af1cf10-d095-4ce0-8edd-7aacf04c698b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68caab932ca04287a3da8319fe815adf', 'request_id': 'req-7af1cf10-d095-4ce0-8edd-7aacf04c698b'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:23.125779 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-7af1cf10-d095-4ce0-8edd-7aacf04c698b admin admin] The token cache is being invalidate because project 68caab932ca04287a3da8319fe815adf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80153) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:00:23.126160 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-7af1cf10-d095-4ce0-8edd-7aacf04c698b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:23.126920 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-7af1cf10-d095-4ce0-8edd-7aacf04c698b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:23.128710 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 947/1889] 10.222.0.64 () {58 vars in 1233 bytes} [Tue Aug 2 10:00:22 2022] DELETE /identity/v3/projects/68caab932ca04287a3da8319fe815adf => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:00:23.133378 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124c6a61-688d-4d7f-89f0-5cc0e5fb5f3b None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:23.133851 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124c6a61-688d-4d7f-89f0-5cc0e5fb5f3b None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:23.133941 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124c6a61-688d-4d7f-89f0-5cc0e5fb5f3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:23.154596 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-124c6a61-688d-4d7f-89f0-5cc0e5fb5f3b None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:00:23.155877 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-124c6a61-688d-4d7f-89f0-5cc0e5fb5f3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:23.228007 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 943/1890] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:00:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:00:23.233305 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4380cb75-6beb-4846-bcc0-f54b2273e21d None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:23.233604 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4380cb75-6beb-4846-bcc0-f54b2273e21d None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:23.233884 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4380cb75-6beb-4846-bcc0-f54b2273e21d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:23.254312 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-4380cb75-6beb-4846-bcc0-f54b2273e21d None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:00:23.255549 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-4380cb75-6beb-4846-bcc0-f54b2273e21d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:23.263639 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 948/1891] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:00:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:00:23.267868 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124c6a61-688d-4d7f-89f0-5cc0e5fb5f3b None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:23.278248 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124c6a61-688d-4d7f-89f0-5cc0e5fb5f3b None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:23.283224 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e24d3b-fe1d-4422-b782-9dc4161eb1bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:23.283907 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e24d3b-fe1d-4422-b782-9dc4161eb1bc admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:23.284190 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e24d3b-fe1d-4422-b782-9dc4161eb1bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:23.284495 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e24d3b-fe1d-4422-b782-9dc4161eb1bc admin admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:23.285213 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e24d3b-fe1d-4422-b782-9dc4161eb1bc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:00:23.285519 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e24d3b-fe1d-4422-b782-9dc4161eb1bc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:23.287090 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e24d3b-fe1d-4422-b782-9dc4161eb1bc admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:23.298342 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 944/1892] 10.222.0.64 () {58 vars in 1153 bytes} [Tue Aug 2 10:00:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:23.302982 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4380cb75-6beb-4846-bcc0-f54b2273e21d None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:23.312465 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4380cb75-6beb-4846-bcc0-f54b2273e21d None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:23.317622 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b254df-452d-4b9a-9c18-b3d76c694894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:23.318234 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b254df-452d-4b9a-9c18-b3d76c694894 admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:23.318520 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b254df-452d-4b9a-9c18-b3d76c694894 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:23.318796 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b254df-452d-4b9a-9c18-b3d76c694894 admin admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:23.319463 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b254df-452d-4b9a-9c18-b3d76c694894 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:23.321431 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b254df-452d-4b9a-9c18-b3d76c694894 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:23.348837 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 949/1893] 10.222.0.64 () {60 vars in 1153 bytes} [Tue Aug 2 10:00:23 2022] POST /identity/v3/projects => generated 375 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:00:23.352898 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e24d3b-fe1d-4422-b782-9dc4161eb1bc admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:23.362796 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e24d3b-fe1d-4422-b782-9dc4161eb1bc admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:23.408623 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:23.422864 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9cde4b-376c-45fb-8403-07dd8883e7dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:23.423598 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9cde4b-376c-45fb-8403-07dd8883e7dd admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:23.424015 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9cde4b-376c-45fb-8403-07dd8883e7dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:23.424460 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9cde4b-376c-45fb-8403-07dd8883e7dd admin admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:23.426851 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9cde4b-376c-45fb-8403-07dd8883e7dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:23.427721 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9cde4b-376c-45fb-8403-07dd8883e7dd admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:23.460747 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 945/1894] 10.222.0.64 () {60 vars in 1144 bytes} [Tue Aug 2 10:00:23 2022] POST /identity/v3/users => generated 288 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:00:23.465480 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b254df-452d-4b9a-9c18-b3d76c694894 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:23.482652 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b254df-452d-4b9a-9c18-b3d76c694894 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:23.488793 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1f4cde-705e-4524-b7df-5d40d057baed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:23.489516 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1f4cde-705e-4524-b7df-5d40d057baed admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:23.489804 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1f4cde-705e-4524-b7df-5d40d057baed admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:23.490231 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1f4cde-705e-4524-b7df-5d40d057baed admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:23.491494 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1f4cde-705e-4524-b7df-5d40d057baed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:23.494353 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1f4cde-705e-4524-b7df-5d40d057baed admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:23.511898 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 950/1895] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:00:23 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:00:23.513045 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9cde4b-376c-45fb-8403-07dd8883e7dd admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:23.527077 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9cde4b-376c-45fb-8403-07dd8883e7dd admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:23.531950 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e37e7ed-b27e-4b12-b9ef-de3bb0931928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:23.532600 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e37e7ed-b27e-4b12-b9ef-de3bb0931928 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:23.532886 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e37e7ed-b27e-4b12-b9ef-de3bb0931928 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:23.533162 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e37e7ed-b27e-4b12-b9ef-de3bb0931928 admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:23.533927 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e37e7ed-b27e-4b12-b9ef-de3bb0931928 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:23.535726 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e37e7ed-b27e-4b12-b9ef-de3bb0931928 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:23.547226 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 946/1896] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:00:23 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:00:23.551828 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1f4cde-705e-4524-b7df-5d40d057baed admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:23.562235 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1f4cde-705e-4524-b7df-5d40d057baed admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:23.569005 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf9bee1-c69f-40c4-9c92-c5680c9c74ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:23.569686 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf9bee1-c69f-40c4-9c92-c5680c9c74ab admin admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:23.569974 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf9bee1-c69f-40c4-9c92-c5680c9c74ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:23.570254 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf9bee1-c69f-40c4-9c92-c5680c9c74ab admin admin] PATH_INFO: `/v3/projects/b1c3178611b249a990bb7b75c4f1259a/users/c2542a28325a4e258ecce94c43f39382/roles/66c1074780954f8e87ed11a5c4e8c36a` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:23.586172 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf9bee1-c69f-40c4-9c92-c5680c9c74ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1c3178611b249a990bb7b75c4f1259a, user_id=c2542a28325a4e258ecce94c43f39382, role_id=66c1074780954f8e87ed11a5c4e8c36a)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:23.588877 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf9bee1-c69f-40c4-9c92-c5680c9c74ab admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:23.604049 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 951/1897] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 10:00:23 2022] PUT /identity/v3/projects/b1c3178611b249a990bb7b75c4f1259a/users/c2542a28325a4e258ecce94c43f39382/roles/66c1074780954f8e87ed11a5c4e8c36a => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:00:23.615939 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cf5420-c08e-4f99-954f-9f838e194c53 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:23.616234 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cf5420-c08e-4f99-954f-9f838e194c53 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:23.616566 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cf5420-c08e-4f99-954f-9f838e194c53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:23.656217 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-46cf5420-c08e-4f99-954f-9f838e194c53 None None] MFA Rules not processed for user `c2542a28325a4e258ecce94c43f39382`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:00:23.657399 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-46cf5420-c08e-4f99-954f-9f838e194c53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:23.725942 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 947/1898] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:00:23 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:00:23.737249 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf9bee1-c69f-40c4-9c92-c5680c9c74ab admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:23.739014 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-4cf9bee1-c69f-40c4-9c92-c5680c9c74ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:23.821820 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf9bee1-c69f-40c4-9c92-c5680c9c74ab admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:23.825737 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9c7f31-3810-4385-8f04-b9968f73fbe3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d7de62b9ee4039bdc79627ac7fb499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:23.826357 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9c7f31-3810-4385-8f04-b9968f73fbe3 service cinder] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:23.826642 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9c7f31-3810-4385-8f04-b9968f73fbe3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:23.826948 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9c7f31-3810-4385-8f04-b9968f73fbe3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:23.844730 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9c7f31-3810-4385-8f04-b9968f73fbe3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:23.846956 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9c7f31-3810-4385-8f04-b9968f73fbe3 service cinder] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:23.858033 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 952/1899] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:00:23 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:00:26.003349 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f0f83d-1ffb-4835-9897-df672c5aa417 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:26.003789 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f0f83d-1ffb-4835-9897-df672c5aa417 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:26.004544 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f0f83d-1ffb-4835-9897-df672c5aa417 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:26.005434 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 948/1900] 10.222.0.64 () {54 vars in 921 bytes} [Tue Aug 2 10:00:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:00:26.025253 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df9c467-d134-4a10-bb6c-a54058c3e201 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:26.025253 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df9c467-d134-4a10-bb6c-a54058c3e201 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:26.026605 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df9c467-d134-4a10-bb6c-a54058c3e201 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:26.072586 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-4df9c467-d134-4a10-bb6c-a54058c3e201 None None] MFA Rules not processed for user `99655e1f9ca24472a86cd26b0dbd1fc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:00:26.091665 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-4df9c467-d134-4a10-bb6c-a54058c3e201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:26.133345 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:26.179381 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 953/1901] 10.222.0.64 () {58 vars in 1018 bytes} [Tue Aug 2 10:00:26 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:00:26.237212 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f0f83d-1ffb-4835-9897-df672c5aa417 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:26.241597 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-99f0f83d-1ffb-4835-9897-df672c5aa417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:26.261694 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f0f83d-1ffb-4835-9897-df672c5aa417 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:26.267264 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9993e3c-b9a1-45aa-b13e-95697f0b8324 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99655e1f9ca24472a86cd26b0dbd1fc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:26.268578 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9993e3c-b9a1-45aa-b13e-95697f0b8324 service nova] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:26.268947 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9993e3c-b9a1-45aa-b13e-95697f0b8324 service nova] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:26.269222 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9993e3c-b9a1-45aa-b13e-95697f0b8324 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:26.279406 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9993e3c-b9a1-45aa-b13e-95697f0b8324 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:26.281024 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9993e3c-b9a1-45aa-b13e-95697f0b8324 service nova] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:26.292302 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 949/1902] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:00:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:00:33.287463 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df9c467-d134-4a10-bb6c-a54058c3e201 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:33.378254 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df9c467-d134-4a10-bb6c-a54058c3e201 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:33.383130 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268e9aba-505f-48d4-814f-af99d312ad8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:33.383851 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268e9aba-505f-48d4-814f-af99d312ad8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:33.384188 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268e9aba-505f-48d4-814f-af99d312ad8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:33.384497 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268e9aba-505f-48d4-814f-af99d312ad8c admin admin] PATH_INFO: `/v3/users/c2542a28325a4e258ecce94c43f39382` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:33.386467 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268e9aba-505f-48d4-814f-af99d312ad8c admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2542a28325a4e258ecce94c43f39382)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:33.388733 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268e9aba-505f-48d4-814f-af99d312ad8c admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:33.448543 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-268e9aba-505f-48d4-814f-af99d312ad8c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2542a28325a4e258ecce94c43f39382'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:33.459530 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-268e9aba-505f-48d4-814f-af99d312ad8c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2542a28325a4e258ecce94c43f39382'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:33.469977 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-268e9aba-505f-48d4-814f-af99d312ad8c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2542a28325a4e258ecce94c43f39382'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:33.471609 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-268e9aba-505f-48d4-814f-af99d312ad8c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2542a28325a4e258ecce94c43f39382'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:33.472309 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-268e9aba-505f-48d4-814f-af99d312ad8c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2542a28325a4e258ecce94c43f39382'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:33.494476 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 954/1903] 10.222.0.64 () {58 vars in 1224 bytes} [Tue Aug 2 10:00:33 2022] DELETE /identity/v3/users/c2542a28325a4e258ecce94c43f39382 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:00:33.496164 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9161e8-d7d6-49ac-b35a-8d38dab5a721 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:33.496748 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9161e8-d7d6-49ac-b35a-8d38dab5a721 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:33.496861 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9161e8-d7d6-49ac-b35a-8d38dab5a721 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:33.518924 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-ff9161e8-d7d6-49ac-b35a-8d38dab5a721 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:00:33.520136 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-ff9161e8-d7d6-49ac-b35a-8d38dab5a721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:33.612780 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 950/1904] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:00:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:00:33.623556 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268e9aba-505f-48d4-814f-af99d312ad8c admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:33.626348 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-268e9aba-505f-48d4-814f-af99d312ad8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:33.720561 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:33.731335 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268e9aba-505f-48d4-814f-af99d312ad8c admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:33.749532 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46320f94-4d1f-4b0d-93b8-f59c5aad513f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:33.750835 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46320f94-4d1f-4b0d-93b8-f59c5aad513f service neutron] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:33.751085 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46320f94-4d1f-4b0d-93b8-f59c5aad513f service neutron] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:33.751789 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46320f94-4d1f-4b0d-93b8-f59c5aad513f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:33.766659 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46320f94-4d1f-4b0d-93b8-f59c5aad513f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:33.769335 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46320f94-4d1f-4b0d-93b8-f59c5aad513f service neutron] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:33.787535 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 955/1905] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:00:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:00:35.284676 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9161e8-d7d6-49ac-b35a-8d38dab5a721 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:35.286918 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-ff9161e8-d7d6-49ac-b35a-8d38dab5a721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:35.301181 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9161e8-d7d6-49ac-b35a-8d38dab5a721 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:35.310153 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42d7639-ba19-4adc-915a-713704d3a806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:35.311168 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42d7639-ba19-4adc-915a-713704d3a806 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:35.311683 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42d7639-ba19-4adc-915a-713704d3a806 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:35.312197 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42d7639-ba19-4adc-915a-713704d3a806 admin admin] PATH_INFO: `/v3/projects/b1c3178611b249a990bb7b75c4f1259a` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:35.313839 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42d7639-ba19-4adc-915a-713704d3a806 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1c3178611b249a990bb7b75c4f1259a)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:35.316682 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42d7639-ba19-4adc-915a-713704d3a806 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:35.429868 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:35.440691 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-b42d7639-ba19-4adc-915a-713704d3a806 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1c3178611b249a990bb7b75c4f1259a', 'request_id': 'req-b42d7639-ba19-4adc-915a-713704d3a806'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:35.449065 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-b42d7639-ba19-4adc-915a-713704d3a806 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1c3178611b249a990bb7b75c4f1259a', 'request_id': 'req-b42d7639-ba19-4adc-915a-713704d3a806'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:35.457262 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-b42d7639-ba19-4adc-915a-713704d3a806 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1c3178611b249a990bb7b75c4f1259a', 'request_id': 'req-b42d7639-ba19-4adc-915a-713704d3a806'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:35.473853 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-b42d7639-ba19-4adc-915a-713704d3a806 admin admin] The token cache is being invalidate because project b1c3178611b249a990bb7b75c4f1259a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80154) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:00:35.474156 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-b42d7639-ba19-4adc-915a-713704d3a806 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:35.474886 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-b42d7639-ba19-4adc-915a-713704d3a806 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:35.480562 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 951/1906] 10.222.0.64 () {58 vars in 1233 bytes} [Tue Aug 2 10:00:35 2022] DELETE /identity/v3/projects/b1c3178611b249a990bb7b75c4f1259a => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:00:35.482213 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cba05f-534a-49f0-bc43-47f7380fe620 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:35.482702 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cba05f-534a-49f0-bc43-47f7380fe620 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:35.482792 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cba05f-534a-49f0-bc43-47f7380fe620 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:35.504015 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-13cba05f-534a-49f0-bc43-47f7380fe620 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:00:35.505749 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-13cba05f-534a-49f0-bc43-47f7380fe620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:35.650566 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 956/1907] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:00:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:00:35.659364 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e57b82-72da-473c-8b82-a0963bbf98bb None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:35.659669 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e57b82-72da-473c-8b82-a0963bbf98bb None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:35.660116 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e57b82-72da-473c-8b82-a0963bbf98bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:35.688208 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-e0e57b82-72da-473c-8b82-a0963bbf98bb None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:00:35.690096 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-e0e57b82-72da-473c-8b82-a0963bbf98bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:35.698436 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 952/1908] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:00:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:00:35.702778 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cba05f-534a-49f0-bc43-47f7380fe620 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:35.713341 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cba05f-534a-49f0-bc43-47f7380fe620 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:35.718105 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa1e466-cd8a-48b7-b091-abc48017c8fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:35.718805 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa1e466-cd8a-48b7-b091-abc48017c8fb admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:35.719163 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa1e466-cd8a-48b7-b091-abc48017c8fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:35.719506 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa1e466-cd8a-48b7-b091-abc48017c8fb admin admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:35.720474 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa1e466-cd8a-48b7-b091-abc48017c8fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:00:35.720885 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa1e466-cd8a-48b7-b091-abc48017c8fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:35.722558 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa1e466-cd8a-48b7-b091-abc48017c8fb admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:35.735063 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 957/1909] 10.222.0.64 () {58 vars in 1153 bytes} [Tue Aug 2 10:00:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:35.739692 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e57b82-72da-473c-8b82-a0963bbf98bb None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:35.749530 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e57b82-72da-473c-8b82-a0963bbf98bb None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:35.756846 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59193b4-0309-4195-8e60-9a7c8df18bde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:35.757611 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59193b4-0309-4195-8e60-9a7c8df18bde admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:35.757995 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59193b4-0309-4195-8e60-9a7c8df18bde admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:35.758366 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59193b4-0309-4195-8e60-9a7c8df18bde admin admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:35.759137 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59193b4-0309-4195-8e60-9a7c8df18bde admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:35.761190 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59193b4-0309-4195-8e60-9a7c8df18bde admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:35.797704 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 953/1910] 10.222.0.64 () {60 vars in 1153 bytes} [Tue Aug 2 10:00:35 2022] POST /identity/v3/projects => generated 385 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:00:35.802542 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa1e466-cd8a-48b7-b091-abc48017c8fb admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:35.812975 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa1e466-cd8a-48b7-b091-abc48017c8fb admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:35.882933 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257766da-85a1-4463-85d5-8a1938a33da6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:35.883862 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257766da-85a1-4463-85d5-8a1938a33da6 admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:35.884158 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257766da-85a1-4463-85d5-8a1938a33da6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:35.884485 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257766da-85a1-4463-85d5-8a1938a33da6 admin admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:35.885186 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257766da-85a1-4463-85d5-8a1938a33da6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:35.887143 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257766da-85a1-4463-85d5-8a1938a33da6 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:35.911695 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 958/1911] 10.222.0.64 () {60 vars in 1144 bytes} [Tue Aug 2 10:00:35 2022] POST /identity/v3/users => generated 293 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:00:35.912826 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59193b4-0309-4195-8e60-9a7c8df18bde admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:35.924544 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59193b4-0309-4195-8e60-9a7c8df18bde admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:35.929692 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8d6636-9de7-4131-a840-3e61ef24fd67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:35.930372 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8d6636-9de7-4131-a840-3e61ef24fd67 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:35.930662 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8d6636-9de7-4131-a840-3e61ef24fd67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:35.930937 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8d6636-9de7-4131-a840-3e61ef24fd67 admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:35.931652 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8d6636-9de7-4131-a840-3e61ef24fd67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:35.933377 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8d6636-9de7-4131-a840-3e61ef24fd67 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:35.945394 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 954/1912] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:00:35 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:00:35.949485 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257766da-85a1-4463-85d5-8a1938a33da6 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:35.962000 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257766da-85a1-4463-85d5-8a1938a33da6 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:35.967506 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1404dd7e-8050-46e0-9f56-3430d6ad6802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:35.968125 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1404dd7e-8050-46e0-9f56-3430d6ad6802 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:35.968444 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1404dd7e-8050-46e0-9f56-3430d6ad6802 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:35.968721 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1404dd7e-8050-46e0-9f56-3430d6ad6802 admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:35.969424 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1404dd7e-8050-46e0-9f56-3430d6ad6802 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:35.971232 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1404dd7e-8050-46e0-9f56-3430d6ad6802 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:35.982980 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 959/1913] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:00:35 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:00:35.987281 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8d6636-9de7-4131-a840-3e61ef24fd67 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:35.998492 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8d6636-9de7-4131-a840-3e61ef24fd67 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:36.003637 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b31ab8-767d-443e-a6d3-21ee6bd689e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:36.004362 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b31ab8-767d-443e-a6d3-21ee6bd689e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:36.004648 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b31ab8-767d-443e-a6d3-21ee6bd689e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:36.004925 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b31ab8-767d-443e-a6d3-21ee6bd689e0 admin admin] PATH_INFO: `/v3/projects/8990416955af4312bc09c5212b38fd42/users/64326b5151a0472c8bf310082b2ca20d/roles/66c1074780954f8e87ed11a5c4e8c36a` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:36.021620 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b31ab8-767d-443e-a6d3-21ee6bd689e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8990416955af4312bc09c5212b38fd42, user_id=64326b5151a0472c8bf310082b2ca20d, role_id=66c1074780954f8e87ed11a5c4e8c36a)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:36.024296 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b31ab8-767d-443e-a6d3-21ee6bd689e0 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:36.037128 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 955/1914] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 10:00:35 2022] PUT /identity/v3/projects/8990416955af4312bc09c5212b38fd42/users/64326b5151a0472c8bf310082b2ca20d/roles/66c1074780954f8e87ed11a5c4e8c36a => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:00:36.047295 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439c699a-235f-4628-b7a4-64cf52b83dfd None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:36.047588 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439c699a-235f-4628-b7a4-64cf52b83dfd None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:36.047868 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439c699a-235f-4628-b7a4-64cf52b83dfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:36.074986 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-439c699a-235f-4628-b7a4-64cf52b83dfd None None] MFA Rules not processed for user `64326b5151a0472c8bf310082b2ca20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:00:36.076221 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-439c699a-235f-4628-b7a4-64cf52b83dfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:36.118685 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:36.139967 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 960/1915] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:00:36 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:00:36.145078 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b31ab8-767d-443e-a6d3-21ee6bd689e0 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:36.208688 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b31ab8-767d-443e-a6d3-21ee6bd689e0 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:36.213427 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa252bee-08e7-4538-ab06-a9eb15b3f33d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:36.214057 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa252bee-08e7-4538-ab06-a9eb15b3f33d admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:36.214343 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa252bee-08e7-4538-ab06-a9eb15b3f33d admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:36.214622 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa252bee-08e7-4538-ab06-a9eb15b3f33d admin admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:36.215291 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa252bee-08e7-4538-ab06-a9eb15b3f33d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:36.217239 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa252bee-08e7-4538-ab06-a9eb15b3f33d admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:36.246082 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 956/1916] 10.222.0.64 () {60 vars in 1153 bytes} [Tue Aug 2 10:00:36 2022] POST /identity/v3/projects => generated 385 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:00:36.250085 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439c699a-235f-4628-b7a4-64cf52b83dfd None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:36.261868 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439c699a-235f-4628-b7a4-64cf52b83dfd None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:36.320226 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2f497a-eed5-464a-bc73-a35e8fe28703 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:36.320873 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2f497a-eed5-464a-bc73-a35e8fe28703 admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:36.321157 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2f497a-eed5-464a-bc73-a35e8fe28703 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:36.321429 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2f497a-eed5-464a-bc73-a35e8fe28703 admin admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:36.322085 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2f497a-eed5-464a-bc73-a35e8fe28703 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:36.324068 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2f497a-eed5-464a-bc73-a35e8fe28703 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:36.345590 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 961/1917] 10.222.0.64 () {60 vars in 1144 bytes} [Tue Aug 2 10:00:36 2022] POST /identity/v3/users => generated 293 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:00:36.349542 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa252bee-08e7-4538-ab06-a9eb15b3f33d admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:36.361050 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa252bee-08e7-4538-ab06-a9eb15b3f33d admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:36.366226 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1a4fad-1c05-445b-8c8e-d4c39cd95ae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:36.366833 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1a4fad-1c05-445b-8c8e-d4c39cd95ae9 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:36.367118 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1a4fad-1c05-445b-8c8e-d4c39cd95ae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:36.367392 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1a4fad-1c05-445b-8c8e-d4c39cd95ae9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:36.368089 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1a4fad-1c05-445b-8c8e-d4c39cd95ae9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:36.369759 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1a4fad-1c05-445b-8c8e-d4c39cd95ae9 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:36.380700 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 957/1918] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:00:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:00:36.384451 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2f497a-eed5-464a-bc73-a35e8fe28703 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:36.394289 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2f497a-eed5-464a-bc73-a35e8fe28703 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:36.398831 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57bd082-2661-4ea1-8d54-acbe596c3cf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:36.399433 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57bd082-2661-4ea1-8d54-acbe596c3cf0 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:36.399723 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57bd082-2661-4ea1-8d54-acbe596c3cf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:36.400002 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57bd082-2661-4ea1-8d54-acbe596c3cf0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:36.400738 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57bd082-2661-4ea1-8d54-acbe596c3cf0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:36.402327 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57bd082-2661-4ea1-8d54-acbe596c3cf0 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:36.412057 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 962/1919] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:00:36 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:00:36.415965 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1a4fad-1c05-445b-8c8e-d4c39cd95ae9 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:36.426616 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1a4fad-1c05-445b-8c8e-d4c39cd95ae9 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:36.432060 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5be1ec-d0f2-450a-86e1-fb8aadf1b13a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:36.432836 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5be1ec-d0f2-450a-86e1-fb8aadf1b13a admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:36.433123 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5be1ec-d0f2-450a-86e1-fb8aadf1b13a admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:36.433397 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5be1ec-d0f2-450a-86e1-fb8aadf1b13a admin admin] PATH_INFO: `/v3/projects/106f1f7c13194382acf736be0d8c9767/users/9be10f070ff2464da4e1d6aa7707eeaf/roles/66c1074780954f8e87ed11a5c4e8c36a` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:36.450749 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5be1ec-d0f2-450a-86e1-fb8aadf1b13a admin admin] RBAC: Authorizing `identity:create_grant(project_id=106f1f7c13194382acf736be0d8c9767, user_id=9be10f070ff2464da4e1d6aa7707eeaf, role_id=66c1074780954f8e87ed11a5c4e8c36a)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:36.453759 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5be1ec-d0f2-450a-86e1-fb8aadf1b13a admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:36.466009 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 958/1920] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 10:00:36 2022] PUT /identity/v3/projects/106f1f7c13194382acf736be0d8c9767/users/9be10f070ff2464da4e1d6aa7707eeaf/roles/66c1074780954f8e87ed11a5c4e8c36a => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:00:36.477101 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc56921-9afd-4f64-a076-72bd890ad1d3 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:36.477394 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc56921-9afd-4f64-a076-72bd890ad1d3 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:36.477673 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc56921-9afd-4f64-a076-72bd890ad1d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:36.506856 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-6bc56921-9afd-4f64-a076-72bd890ad1d3 None None] MFA Rules not processed for user `9be10f070ff2464da4e1d6aa7707eeaf`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:00:36.508068 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-6bc56921-9afd-4f64-a076-72bd890ad1d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:36.573077 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 963/1921] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:00:36 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:00:36.577992 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5be1ec-d0f2-450a-86e1-fb8aadf1b13a admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:36.592262 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:36.661243 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5be1ec-d0f2-450a-86e1-fb8aadf1b13a admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:36.668149 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e140a991-9002-4e96-8554-86b9a6100fbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:36.668896 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e140a991-9002-4e96-8554-86b9a6100fbd admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:36.669254 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e140a991-9002-4e96-8554-86b9a6100fbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:36.669601 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e140a991-9002-4e96-8554-86b9a6100fbd admin admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:36.670375 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e140a991-9002-4e96-8554-86b9a6100fbd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:36.672581 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e140a991-9002-4e96-8554-86b9a6100fbd admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:36.710118 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 959/1922] 10.222.0.64 () {60 vars in 1153 bytes} [Tue Aug 2 10:00:36 2022] POST /identity/v3/projects => generated 385 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:00:36.710509 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc56921-9afd-4f64-a076-72bd890ad1d3 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:36.726596 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc56921-9afd-4f64-a076-72bd890ad1d3 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:36.786123 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:36.798169 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53b8e47-6b9d-4f10-9e7c-6d95671aeda9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:36.799398 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53b8e47-6b9d-4f10-9e7c-6d95671aeda9 admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:36.800012 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53b8e47-6b9d-4f10-9e7c-6d95671aeda9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:36.800717 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53b8e47-6b9d-4f10-9e7c-6d95671aeda9 admin admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:36.801399 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53b8e47-6b9d-4f10-9e7c-6d95671aeda9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:36.803332 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53b8e47-6b9d-4f10-9e7c-6d95671aeda9 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:36.824930 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 964/1923] 10.222.0.64 () {60 vars in 1144 bytes} [Tue Aug 2 10:00:36 2022] POST /identity/v3/users => generated 299 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:00:36.829255 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e140a991-9002-4e96-8554-86b9a6100fbd admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:36.841047 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e140a991-9002-4e96-8554-86b9a6100fbd admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:36.850423 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe2582c-ecaf-454a-a36f-57540ddb0390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:36.851191 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe2582c-ecaf-454a-a36f-57540ddb0390 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:36.851582 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe2582c-ecaf-454a-a36f-57540ddb0390 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:36.851924 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe2582c-ecaf-454a-a36f-57540ddb0390 admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:36.852734 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe2582c-ecaf-454a-a36f-57540ddb0390 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:36.854455 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe2582c-ecaf-454a-a36f-57540ddb0390 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:36.865395 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 960/1924] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:00:36 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:00:36.869889 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53b8e47-6b9d-4f10-9e7c-6d95671aeda9 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:36.880410 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53b8e47-6b9d-4f10-9e7c-6d95671aeda9 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:36.885119 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d178b70-d55a-4fc7-a943-5097722cb63c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:36.885799 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d178b70-d55a-4fc7-a943-5097722cb63c admin admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:36.886083 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d178b70-d55a-4fc7-a943-5097722cb63c admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:36.886362 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d178b70-d55a-4fc7-a943-5097722cb63c admin admin] PATH_INFO: `/v3/projects/a006b02764ac4027acae27f0e9022dc5/users/c69c032f01174a51971cc8bc66cd73e0/roles/1000cf318fa14915b8091599eaead5de` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:36.905583 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d178b70-d55a-4fc7-a943-5097722cb63c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a006b02764ac4027acae27f0e9022dc5, user_id=c69c032f01174a51971cc8bc66cd73e0, role_id=1000cf318fa14915b8091599eaead5de)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:36.908218 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d178b70-d55a-4fc7-a943-5097722cb63c admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:36.922941 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 965/1925] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 10:00:36 2022] PUT /identity/v3/projects/a006b02764ac4027acae27f0e9022dc5/users/c69c032f01174a51971cc8bc66cd73e0/roles/1000cf318fa14915b8091599eaead5de => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:00:36.927118 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe2582c-ecaf-454a-a36f-57540ddb0390 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:36.937485 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe2582c-ecaf-454a-a36f-57540ddb0390 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:36.993837 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a40630-776e-45ac-b622-bac53f7b3ec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:36.994459 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a40630-776e-45ac-b622-bac53f7b3ec5 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:36.994742 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a40630-776e-45ac-b622-bac53f7b3ec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:36.995017 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a40630-776e-45ac-b622-bac53f7b3ec5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:36.995839 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a40630-776e-45ac-b622-bac53f7b3ec5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:36.997468 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a40630-776e-45ac-b622-bac53f7b3ec5 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:37.007766 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 961/1926] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:00:36 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:00:37.011970 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d178b70-d55a-4fc7-a943-5097722cb63c admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:37.023037 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d178b70-d55a-4fc7-a943-5097722cb63c admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:37.027992 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e5f3d6-ad8e-4407-97fd-c2225974db78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:37.028920 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e5f3d6-ad8e-4407-97fd-c2225974db78 admin admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:37.029271 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e5f3d6-ad8e-4407-97fd-c2225974db78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:37.031282 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e5f3d6-ad8e-4407-97fd-c2225974db78 admin admin] PATH_INFO: `/v3/projects/a006b02764ac4027acae27f0e9022dc5/users/c69c032f01174a51971cc8bc66cd73e0/roles/1000cf318fa14915b8091599eaead5de` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:37.031945 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e5f3d6-ad8e-4407-97fd-c2225974db78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a006b02764ac4027acae27f0e9022dc5, user_id=c69c032f01174a51971cc8bc66cd73e0, role_id=1000cf318fa14915b8091599eaead5de)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:37.035217 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e5f3d6-ad8e-4407-97fd-c2225974db78 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:37.045887 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 966/1927] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 10:00:37 2022] PUT /identity/v3/projects/a006b02764ac4027acae27f0e9022dc5/users/c69c032f01174a51971cc8bc66cd73e0/roles/1000cf318fa14915b8091599eaead5de => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:00:37.056792 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7fe30d-0a9f-4f1f-8a16-1d448449352c None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:37.057088 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7fe30d-0a9f-4f1f-8a16-1d448449352c None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:37.057370 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7fe30d-0a9f-4f1f-8a16-1d448449352c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:37.085550 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-fb7fe30d-0a9f-4f1f-8a16-1d448449352c None None] MFA Rules not processed for user `c69c032f01174a51971cc8bc66cd73e0`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:00:37.086778 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-fb7fe30d-0a9f-4f1f-8a16-1d448449352c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:37.159374 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 962/1928] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:00:37 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:00:37.170473 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e5f3d6-ad8e-4407-97fd-c2225974db78 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:37.172167 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-d0e5f3d6-ad8e-4407-97fd-c2225974db78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:37.229721 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e5f3d6-ad8e-4407-97fd-c2225974db78 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:37.233294 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1b850b-c0e5-4def-8b52-afccabe0647b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d7de62b9ee4039bdc79627ac7fb499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:37.233902 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1b850b-c0e5-4def-8b52-afccabe0647b service cinder] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:37.234184 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1b850b-c0e5-4def-8b52-afccabe0647b service cinder] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:37.234457 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1b850b-c0e5-4def-8b52-afccabe0647b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:37.290295 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1b850b-c0e5-4def-8b52-afccabe0647b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:37.291736 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1b850b-c0e5-4def-8b52-afccabe0647b service cinder] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:37.300474 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 967/1929] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:00:37 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:00:43.177708 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7fe30d-0a9f-4f1f-8a16-1d448449352c None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:43.179234 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-fb7fe30d-0a9f-4f1f-8a16-1d448449352c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:43.213159 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7fe30d-0a9f-4f1f-8a16-1d448449352c None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:43.221803 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6828891-b291-4024-9542-8573e6090311 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d7de62b9ee4039bdc79627ac7fb499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:43.222805 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6828891-b291-4024-9542-8573e6090311 service cinder] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:43.223348 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6828891-b291-4024-9542-8573e6090311 service cinder] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:43.224665 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6828891-b291-4024-9542-8573e6090311 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:43.233136 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:43.328977 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6828891-b291-4024-9542-8573e6090311 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:43.330479 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6828891-b291-4024-9542-8573e6090311 service cinder] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:43.340947 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 963/1930] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:00:43 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:00:44.028647 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1b850b-c0e5-4def-8b52-afccabe0647b service cinder] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:44.037750 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1b850b-c0e5-4def-8b52-afccabe0647b service cinder] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:44.041215 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31702c25-62d6-48e4-aa96-579bcb17fbcc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d7de62b9ee4039bdc79627ac7fb499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:44.041923 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31702c25-62d6-48e4-aa96-579bcb17fbcc service cinder] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:44.042205 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31702c25-62d6-48e4-aa96-579bcb17fbcc service cinder] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:44.042479 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31702c25-62d6-48e4-aa96-579bcb17fbcc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:44.052421 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31702c25-62d6-48e4-aa96-579bcb17fbcc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:44.053896 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31702c25-62d6-48e4-aa96-579bcb17fbcc service cinder] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:44.063355 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 968/1931] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:00:44 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:00:45.869001 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6828891-b291-4024-9542-8573e6090311 service cinder] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:45.870513 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-c6828891-b291-4024-9542-8573e6090311 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:45.939775 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6828891-b291-4024-9542-8573e6090311 service cinder] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:45.944233 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0393e0e-de7d-427d-98b3-0fabba433815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:45.944905 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0393e0e-de7d-427d-98b3-0fabba433815 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:45.945193 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0393e0e-de7d-427d-98b3-0fabba433815 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:45.945474 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0393e0e-de7d-427d-98b3-0fabba433815 admin admin] PATH_INFO: `/v3/users/9d15622994074a699eacc36c6b7d7123` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:45.946663 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0393e0e-de7d-427d-98b3-0fabba433815 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d15622994074a699eacc36c6b7d7123)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:45.948644 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0393e0e-de7d-427d-98b3-0fabba433815 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:46.002765 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-e0393e0e-de7d-427d-98b3-0fabba433815 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d15622994074a699eacc36c6b7d7123'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:46.009963 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-e0393e0e-de7d-427d-98b3-0fabba433815 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d15622994074a699eacc36c6b7d7123'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:46.010765 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-e0393e0e-de7d-427d-98b3-0fabba433815 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d15622994074a699eacc36c6b7d7123'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:46.019214 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-e0393e0e-de7d-427d-98b3-0fabba433815 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d15622994074a699eacc36c6b7d7123'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:46.032272 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-e0393e0e-de7d-427d-98b3-0fabba433815 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d15622994074a699eacc36c6b7d7123'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:46.042931 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 964/1932] 10.222.0.64 () {58 vars in 1224 bytes} [Tue Aug 2 10:00:45 2022] DELETE /identity/v3/users/9d15622994074a699eacc36c6b7d7123 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:00:46.047310 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a070f8ad-22ee-441a-b21a-290cfc5b6cba None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:46.047756 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a070f8ad-22ee-441a-b21a-290cfc5b6cba None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:46.047874 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a070f8ad-22ee-441a-b21a-290cfc5b6cba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:46.074525 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-a070f8ad-22ee-441a-b21a-290cfc5b6cba None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:00:46.076023 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-a070f8ad-22ee-441a-b21a-290cfc5b6cba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:46.121031 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:46.140312 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 969/1933] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:00:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:00:46.150413 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0393e0e-de7d-427d-98b3-0fabba433815 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:46.151888 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-e0393e0e-de7d-427d-98b3-0fabba433815 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:46.185367 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:46.215850 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0393e0e-de7d-427d-98b3-0fabba433815 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:46.219785 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9839914-7b5f-45b0-afa9-47cf4e09a615 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:46.220648 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9839914-7b5f-45b0-afa9-47cf4e09a615 service neutron] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:46.221000 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9839914-7b5f-45b0-afa9-47cf4e09a615 service neutron] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:46.221336 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9839914-7b5f-45b0-afa9-47cf4e09a615 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:46.231438 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9839914-7b5f-45b0-afa9-47cf4e09a615 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:46.233059 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9839914-7b5f-45b0-afa9-47cf4e09a615 service neutron] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:46.243372 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 965/1934] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:00:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:00:46.827808 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a070f8ad-22ee-441a-b21a-290cfc5b6cba None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:46.833682 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-a070f8ad-22ee-441a-b21a-290cfc5b6cba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:46.864113 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a070f8ad-22ee-441a-b21a-290cfc5b6cba None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:46.869585 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142bea3b-73a2-4ba4-9d00-0cac6da3b8e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:46.870681 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142bea3b-73a2-4ba4-9d00-0cac6da3b8e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:46.871439 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142bea3b-73a2-4ba4-9d00-0cac6da3b8e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:46.871657 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142bea3b-73a2-4ba4-9d00-0cac6da3b8e8 admin admin] PATH_INFO: `/v3/projects/c3b8c6297eea426cb29708e0ef857f47` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:46.874738 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142bea3b-73a2-4ba4-9d00-0cac6da3b8e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3b8c6297eea426cb29708e0ef857f47)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:46.877241 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142bea3b-73a2-4ba4-9d00-0cac6da3b8e8 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:46.974758 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-142bea3b-73a2-4ba4-9d00-0cac6da3b8e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3b8c6297eea426cb29708e0ef857f47', 'request_id': 'req-142bea3b-73a2-4ba4-9d00-0cac6da3b8e8'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:46.981045 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-142bea3b-73a2-4ba4-9d00-0cac6da3b8e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3b8c6297eea426cb29708e0ef857f47', 'request_id': 'req-142bea3b-73a2-4ba4-9d00-0cac6da3b8e8'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:46.988525 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-142bea3b-73a2-4ba4-9d00-0cac6da3b8e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3b8c6297eea426cb29708e0ef857f47', 'request_id': 'req-142bea3b-73a2-4ba4-9d00-0cac6da3b8e8'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:47.010605 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-142bea3b-73a2-4ba4-9d00-0cac6da3b8e8 admin admin] The token cache is being invalidate because project c3b8c6297eea426cb29708e0ef857f47 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80153) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:00:47.011093 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-142bea3b-73a2-4ba4-9d00-0cac6da3b8e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:47.011716 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-142bea3b-73a2-4ba4-9d00-0cac6da3b8e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:47.013307 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 970/1935] 10.222.0.64 () {58 vars in 1233 bytes} [Tue Aug 2 10:00:46 2022] DELETE /identity/v3/projects/c3b8c6297eea426cb29708e0ef857f47 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:00:47.017422 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9839914-7b5f-45b0-afa9-47cf4e09a615 service neutron] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:47.027586 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9839914-7b5f-45b0-afa9-47cf4e09a615 service neutron] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:47.095366 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdd873f-5820-4a3f-adfa-bb6ebfa8320f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:47.096170 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdd873f-5820-4a3f-adfa-bb6ebfa8320f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:47.096504 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdd873f-5820-4a3f-adfa-bb6ebfa8320f admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:47.096782 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdd873f-5820-4a3f-adfa-bb6ebfa8320f admin admin] PATH_INFO: `/v3/users/6f20fd3be74e4702b366a1a00a77103f` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:47.098041 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdd873f-5820-4a3f-adfa-bb6ebfa8320f admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f20fd3be74e4702b366a1a00a77103f)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:47.099962 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdd873f-5820-4a3f-adfa-bb6ebfa8320f admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:47.175726 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-5bdd873f-5820-4a3f-adfa-bb6ebfa8320f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f20fd3be74e4702b366a1a00a77103f'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:47.182851 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-5bdd873f-5820-4a3f-adfa-bb6ebfa8320f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f20fd3be74e4702b366a1a00a77103f'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:47.183670 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-5bdd873f-5820-4a3f-adfa-bb6ebfa8320f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f20fd3be74e4702b366a1a00a77103f'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:47.191426 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-5bdd873f-5820-4a3f-adfa-bb6ebfa8320f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f20fd3be74e4702b366a1a00a77103f'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:47.204701 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-5bdd873f-5820-4a3f-adfa-bb6ebfa8320f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f20fd3be74e4702b366a1a00a77103f'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:47.210789 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 966/1936] 10.222.0.64 () {58 vars in 1224 bytes} [Tue Aug 2 10:00:47 2022] DELETE /identity/v3/users/6f20fd3be74e4702b366a1a00a77103f => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:00:47.534667 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142bea3b-73a2-4ba4-9d00-0cac6da3b8e8 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:47.536874 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-142bea3b-73a2-4ba4-9d00-0cac6da3b8e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:47.553431 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142bea3b-73a2-4ba4-9d00-0cac6da3b8e8 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:47.613451 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9a5d55-6d3e-436a-9156-dd9619723230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:47.614134 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9a5d55-6d3e-436a-9156-dd9619723230 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:47.614583 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9a5d55-6d3e-436a-9156-dd9619723230 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:47.614729 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9a5d55-6d3e-436a-9156-dd9619723230 admin admin] PATH_INFO: `/v3/projects/3bf9484227274dcbb4db19c776dd6ad5` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:47.615919 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9a5d55-6d3e-436a-9156-dd9619723230 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3bf9484227274dcbb4db19c776dd6ad5)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:47.617892 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9a5d55-6d3e-436a-9156-dd9619723230 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:47.666626 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-6f9a5d55-6d3e-436a-9156-dd9619723230 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3bf9484227274dcbb4db19c776dd6ad5', 'request_id': 'req-6f9a5d55-6d3e-436a-9156-dd9619723230'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:47.673250 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-6f9a5d55-6d3e-436a-9156-dd9619723230 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3bf9484227274dcbb4db19c776dd6ad5', 'request_id': 'req-6f9a5d55-6d3e-436a-9156-dd9619723230'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:47.680217 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-6f9a5d55-6d3e-436a-9156-dd9619723230 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3bf9484227274dcbb4db19c776dd6ad5', 'request_id': 'req-6f9a5d55-6d3e-436a-9156-dd9619723230'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:47.690424 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:47.695203 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-6f9a5d55-6d3e-436a-9156-dd9619723230 admin admin] The token cache is being invalidate because project 3bf9484227274dcbb4db19c776dd6ad5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80153) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:00:47.695667 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-6f9a5d55-6d3e-436a-9156-dd9619723230 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:47.696482 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-6f9a5d55-6d3e-436a-9156-dd9619723230 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:47.698549 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 971/1937] 10.222.0.64 () {58 vars in 1233 bytes} [Tue Aug 2 10:00:47 2022] DELETE /identity/v3/projects/3bf9484227274dcbb4db19c776dd6ad5 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:00:47.703875 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e45915-037d-440d-abae-be5dc0814b22 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:47.704168 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e45915-037d-440d-abae-be5dc0814b22 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:47.705018 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e45915-037d-440d-abae-be5dc0814b22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:47.725556 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-26e45915-037d-440d-abae-be5dc0814b22 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:00:47.726748 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-26e45915-037d-440d-abae-be5dc0814b22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:47.772915 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:47.792330 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 967/1938] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:00:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:00:47.800400 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ce07ea-9bb7-4a6e-82e6-fd143422a9f0 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:47.801894 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ce07ea-9bb7-4a6e-82e6-fd143422a9f0 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:47.802185 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ce07ea-9bb7-4a6e-82e6-fd143422a9f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:47.824128 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-b9ce07ea-9bb7-4a6e-82e6-fd143422a9f0 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:00:47.825387 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-b9ce07ea-9bb7-4a6e-82e6-fd143422a9f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:47.834465 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 972/1939] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:00:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:00:47.839131 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e45915-037d-440d-abae-be5dc0814b22 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:47.850033 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e45915-037d-440d-abae-be5dc0814b22 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:47.854840 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95d9fde-a3b0-48b1-8186-2166479aa0b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:47.855456 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95d9fde-a3b0-48b1-8186-2166479aa0b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:47.855739 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95d9fde-a3b0-48b1-8186-2166479aa0b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:47.856012 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95d9fde-a3b0-48b1-8186-2166479aa0b1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:47.856750 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95d9fde-a3b0-48b1-8186-2166479aa0b1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:00:47.857054 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95d9fde-a3b0-48b1-8186-2166479aa0b1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:47.858638 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95d9fde-a3b0-48b1-8186-2166479aa0b1 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:47.874432 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 968/1940] 10.222.0.64 () {58 vars in 1153 bytes} [Tue Aug 2 10:00:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:00:47.877753 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ce07ea-9bb7-4a6e-82e6-fd143422a9f0 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:47.893862 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ce07ea-9bb7-4a6e-82e6-fd143422a9f0 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:47.899225 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49ee516-752b-4548-afa3-442e64d3db2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:47.900097 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49ee516-752b-4548-afa3-442e64d3db2b admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:47.900489 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49ee516-752b-4548-afa3-442e64d3db2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:47.900842 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49ee516-752b-4548-afa3-442e64d3db2b admin admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:47.901592 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49ee516-752b-4548-afa3-442e64d3db2b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:47.903809 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49ee516-752b-4548-afa3-442e64d3db2b admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:47.944076 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 973/1941] 10.222.0.64 () {60 vars in 1153 bytes} [Tue Aug 2 10:00:47 2022] POST /identity/v3/projects => generated 387 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:00:47.945528 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95d9fde-a3b0-48b1-8186-2166479aa0b1 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:47.960541 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95d9fde-a3b0-48b1-8186-2166479aa0b1 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:48.026569 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ec42bd-ff15-47a1-ab3f-061a94e2d1b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:48.027273 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ec42bd-ff15-47a1-ab3f-061a94e2d1b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:48.027448 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ec42bd-ff15-47a1-ab3f-061a94e2d1b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:48.027739 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ec42bd-ff15-47a1-ab3f-061a94e2d1b6 admin admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:48.028420 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ec42bd-ff15-47a1-ab3f-061a94e2d1b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:48.030323 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ec42bd-ff15-47a1-ab3f-061a94e2d1b6 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:48.053031 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 969/1942] 10.222.0.64 () {60 vars in 1144 bytes} [Tue Aug 2 10:00:47 2022] POST /identity/v3/users => generated 294 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:00:48.057288 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49ee516-752b-4548-afa3-442e64d3db2b admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:48.068597 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49ee516-752b-4548-afa3-442e64d3db2b admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:48.073299 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5724873-90d8-4cdb-8ca3-71f9abfeb6b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:48.073932 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5724873-90d8-4cdb-8ca3-71f9abfeb6b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:48.074216 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5724873-90d8-4cdb-8ca3-71f9abfeb6b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:48.074491 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5724873-90d8-4cdb-8ca3-71f9abfeb6b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:48.075187 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5724873-90d8-4cdb-8ca3-71f9abfeb6b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:48.077083 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5724873-90d8-4cdb-8ca3-71f9abfeb6b4 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:48.086954 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 974/1943] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:00:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:00:48.090862 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ec42bd-ff15-47a1-ab3f-061a94e2d1b6 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:48.101318 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ec42bd-ff15-47a1-ab3f-061a94e2d1b6 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:48.106354 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3b124e-0f0d-4b72-ab06-af1198216443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:48.106961 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3b124e-0f0d-4b72-ab06-af1198216443 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:48.107249 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3b124e-0f0d-4b72-ab06-af1198216443 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:48.107527 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3b124e-0f0d-4b72-ab06-af1198216443 admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:48.108229 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3b124e-0f0d-4b72-ab06-af1198216443 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:48.109854 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3b124e-0f0d-4b72-ab06-af1198216443 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:48.121743 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 970/1944] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:00:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:00:48.126446 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5724873-90d8-4cdb-8ca3-71f9abfeb6b4 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:48.136755 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5724873-90d8-4cdb-8ca3-71f9abfeb6b4 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:48.141420 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc308143-a0a0-44ed-a35b-923fd368b9e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:48.142170 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc308143-a0a0-44ed-a35b-923fd368b9e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:48.142473 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc308143-a0a0-44ed-a35b-923fd368b9e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:48.142764 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc308143-a0a0-44ed-a35b-923fd368b9e0 admin admin] PATH_INFO: `/v3/projects/a08817dbfccc418bb57f4a1c92aca9be/users/9db4f7224f1c4ee48b634b3d0a9fcfb0/roles/66c1074780954f8e87ed11a5c4e8c36a` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:48.160201 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc308143-a0a0-44ed-a35b-923fd368b9e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a08817dbfccc418bb57f4a1c92aca9be, user_id=9db4f7224f1c4ee48b634b3d0a9fcfb0, role_id=66c1074780954f8e87ed11a5c4e8c36a)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:48.162985 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc308143-a0a0-44ed-a35b-923fd368b9e0 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:48.183938 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 975/1945] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 10:00:48 2022] PUT /identity/v3/projects/a08817dbfccc418bb57f4a1c92aca9be/users/9db4f7224f1c4ee48b634b3d0a9fcfb0/roles/66c1074780954f8e87ed11a5c4e8c36a => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:00:48.196544 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e590dd0-f4df-4b09-bcb9-bd9d62a75a58 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:48.196787 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e590dd0-f4df-4b09-bcb9-bd9d62a75a58 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:48.197093 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e590dd0-f4df-4b09-bcb9-bd9d62a75a58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:48.230297 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-8e590dd0-f4df-4b09-bcb9-bd9d62a75a58 None None] MFA Rules not processed for user `9db4f7224f1c4ee48b634b3d0a9fcfb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:00:48.231529 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-8e590dd0-f4df-4b09-bcb9-bd9d62a75a58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:48.297449 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 971/1946] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:00:48 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:00:48.302039 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc308143-a0a0-44ed-a35b-923fd368b9e0 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:48.382866 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc308143-a0a0-44ed-a35b-923fd368b9e0 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:48.406462 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dd8711-61aa-4528-8600-f09d6dd97de2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:48.407103 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dd8711-61aa-4528-8600-f09d6dd97de2 admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:48.407396 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dd8711-61aa-4528-8600-f09d6dd97de2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:48.407759 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dd8711-61aa-4528-8600-f09d6dd97de2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:48.408843 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dd8711-61aa-4528-8600-f09d6dd97de2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:48.414829 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dd8711-61aa-4528-8600-f09d6dd97de2 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:48.452981 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 976/1947] 10.222.0.64 () {60 vars in 1153 bytes} [Tue Aug 2 10:00:48 2022] POST /identity/v3/projects => generated 387 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:00:48.456907 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e590dd0-f4df-4b09-bcb9-bd9d62a75a58 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:48.469993 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e590dd0-f4df-4b09-bcb9-bd9d62a75a58 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:48.513248 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:48.530714 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3585f69c-f020-4134-ab2c-54e38205027e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:48.531538 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3585f69c-f020-4134-ab2c-54e38205027e admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:48.531885 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3585f69c-f020-4134-ab2c-54e38205027e admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:48.532183 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3585f69c-f020-4134-ab2c-54e38205027e admin admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:48.532943 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3585f69c-f020-4134-ab2c-54e38205027e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:48.534983 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3585f69c-f020-4134-ab2c-54e38205027e admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:48.560690 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 972/1948] 10.222.0.64 () {60 vars in 1144 bytes} [Tue Aug 2 10:00:48 2022] POST /identity/v3/users => generated 300 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:00:48.561620 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dd8711-61aa-4528-8600-f09d6dd97de2 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:48.572577 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dd8711-61aa-4528-8600-f09d6dd97de2 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:48.577916 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f596cd77-3927-4c0f-9c07-425e2065b52c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:48.578605 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f596cd77-3927-4c0f-9c07-425e2065b52c admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:48.578960 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f596cd77-3927-4c0f-9c07-425e2065b52c admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:48.579307 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f596cd77-3927-4c0f-9c07-425e2065b52c admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:48.580172 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f596cd77-3927-4c0f-9c07-425e2065b52c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:48.581953 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f596cd77-3927-4c0f-9c07-425e2065b52c admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:48.593094 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 977/1949] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:00:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:00:48.600441 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3585f69c-f020-4134-ab2c-54e38205027e admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:48.611688 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3585f69c-f020-4134-ab2c-54e38205027e admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:48.616349 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b305a06-878e-49a6-b34d-fff716c6f439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:48.617020 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b305a06-878e-49a6-b34d-fff716c6f439 admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:48.617310 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b305a06-878e-49a6-b34d-fff716c6f439 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:48.617593 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b305a06-878e-49a6-b34d-fff716c6f439 admin admin] PATH_INFO: `/v3/projects/f8db6f6cb4a64bbf8eac38bef9033fde/users/50d3a33a16ad4f72ae232277739757b7/roles/1000cf318fa14915b8091599eaead5de` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:48.632869 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b305a06-878e-49a6-b34d-fff716c6f439 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8db6f6cb4a64bbf8eac38bef9033fde, user_id=50d3a33a16ad4f72ae232277739757b7, role_id=1000cf318fa14915b8091599eaead5de)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:48.635513 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b305a06-878e-49a6-b34d-fff716c6f439 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:48.649209 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 973/1950] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 10:00:48 2022] PUT /identity/v3/projects/f8db6f6cb4a64bbf8eac38bef9033fde/users/50d3a33a16ad4f72ae232277739757b7/roles/1000cf318fa14915b8091599eaead5de => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:00:48.653219 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f596cd77-3927-4c0f-9c07-425e2065b52c admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:48.663796 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f596cd77-3927-4c0f-9c07-425e2065b52c admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:48.722162 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6ffc3d-7b66-4a0e-a6d0-f1c7c982eb36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:48.722840 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6ffc3d-7b66-4a0e-a6d0-f1c7c982eb36 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:48.723126 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6ffc3d-7b66-4a0e-a6d0-f1c7c982eb36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:48.723404 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6ffc3d-7b66-4a0e-a6d0-f1c7c982eb36 admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:48.724135 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6ffc3d-7b66-4a0e-a6d0-f1c7c982eb36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:48.725770 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6ffc3d-7b66-4a0e-a6d0-f1c7c982eb36 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:48.735336 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 978/1951] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:00:48 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:00:48.739752 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b305a06-878e-49a6-b34d-fff716c6f439 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:48.750424 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b305a06-878e-49a6-b34d-fff716c6f439 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:48.755761 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fb0ac9-3e5b-4d53-ae77-a35439677a75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:48.756467 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fb0ac9-3e5b-4d53-ae77-a35439677a75 admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:48.756757 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fb0ac9-3e5b-4d53-ae77-a35439677a75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:48.757037 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fb0ac9-3e5b-4d53-ae77-a35439677a75 admin admin] PATH_INFO: `/v3/projects/f8db6f6cb4a64bbf8eac38bef9033fde/users/50d3a33a16ad4f72ae232277739757b7/roles/1000cf318fa14915b8091599eaead5de` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:48.759018 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fb0ac9-3e5b-4d53-ae77-a35439677a75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8db6f6cb4a64bbf8eac38bef9033fde, user_id=50d3a33a16ad4f72ae232277739757b7, role_id=1000cf318fa14915b8091599eaead5de)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:48.761693 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fb0ac9-3e5b-4d53-ae77-a35439677a75 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:48.773246 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 974/1952] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 10:00:48 2022] PUT /identity/v3/projects/f8db6f6cb4a64bbf8eac38bef9033fde/users/50d3a33a16ad4f72ae232277739757b7/roles/1000cf318fa14915b8091599eaead5de => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:00:48.783686 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c913c1-57fb-4843-88f1-7fb70cd64853 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:48.783980 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c913c1-57fb-4843-88f1-7fb70cd64853 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:48.784305 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c913c1-57fb-4843-88f1-7fb70cd64853 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:48.797987 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:48.812208 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-c6c913c1-57fb-4843-88f1-7fb70cd64853 None None] MFA Rules not processed for user `50d3a33a16ad4f72ae232277739757b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:00:48.813509 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-c6c913c1-57fb-4843-88f1-7fb70cd64853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:48.885889 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 979/1953] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:00:48 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:00:48.898508 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fb0ac9-3e5b-4d53-ae77-a35439677a75 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:48.900374 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-81fb0ac9-3e5b-4d53-ae77-a35439677a75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:48.958332 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fb0ac9-3e5b-4d53-ae77-a35439677a75 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:48.961984 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba161a07-b717-4f74-8500-669d5267b923 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d7de62b9ee4039bdc79627ac7fb499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:48.962593 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba161a07-b717-4f74-8500-669d5267b923 service cinder] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:48.962875 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba161a07-b717-4f74-8500-669d5267b923 service cinder] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:48.963148 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba161a07-b717-4f74-8500-669d5267b923 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:48.972000 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba161a07-b717-4f74-8500-669d5267b923 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:48.973495 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba161a07-b717-4f74-8500-669d5267b923 service cinder] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:48.982644 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 975/1954] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:00:48 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:00:49.051164 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c913c1-57fb-4843-88f1-7fb70cd64853 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:49.087400 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba161a07-b717-4f74-8500-669d5267b923 service cinder] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:49.088980 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-ba161a07-b717-4f74-8500-669d5267b923 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:49.134564 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c913c1-57fb-4843-88f1-7fb70cd64853 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:49.139373 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a690585-bdc4-4ba5-8d0a-110723b4656d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:49.140026 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a690585-bdc4-4ba5-8d0a-110723b4656d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:49.140344 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a690585-bdc4-4ba5-8d0a-110723b4656d admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:49.140626 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a690585-bdc4-4ba5-8d0a-110723b4656d admin admin] PATH_INFO: `/v3/users/9db4f7224f1c4ee48b634b3d0a9fcfb0` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:49.141817 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a690585-bdc4-4ba5-8d0a-110723b4656d admin admin] RBAC: Authorizing `identity:delete_user(user_id=9db4f7224f1c4ee48b634b3d0a9fcfb0)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:49.143806 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a690585-bdc4-4ba5-8d0a-110723b4656d admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:49.159597 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba161a07-b717-4f74-8500-669d5267b923 service cinder] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:49.165663 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a568338a-a0d8-4154-9609-c58c7514264e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:49.166510 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a568338a-a0d8-4154-9609-c58c7514264e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:49.166946 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a568338a-a0d8-4154-9609-c58c7514264e admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:49.167372 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a568338a-a0d8-4154-9609-c58c7514264e admin admin] PATH_INFO: `/v3/users/64326b5151a0472c8bf310082b2ca20d` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:49.168796 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a568338a-a0d8-4154-9609-c58c7514264e admin admin] RBAC: Authorizing `identity:delete_user(user_id=64326b5151a0472c8bf310082b2ca20d)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:49.170714 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a568338a-a0d8-4154-9609-c58c7514264e admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:49.202560 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-2a690585-bdc4-4ba5-8d0a-110723b4656d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9db4f7224f1c4ee48b634b3d0a9fcfb0'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:49.213280 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-2a690585-bdc4-4ba5-8d0a-110723b4656d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9db4f7224f1c4ee48b634b3d0a9fcfb0'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:49.222849 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-2a690585-bdc4-4ba5-8d0a-110723b4656d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9db4f7224f1c4ee48b634b3d0a9fcfb0'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:49.223665 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-2a690585-bdc4-4ba5-8d0a-110723b4656d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9db4f7224f1c4ee48b634b3d0a9fcfb0'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:49.224338 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-2a690585-bdc4-4ba5-8d0a-110723b4656d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9db4f7224f1c4ee48b634b3d0a9fcfb0'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:49.226575 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=89,keystone:INSERT=5,keystone:DELETE=6 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:49.234536 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-a568338a-a0d8-4154-9609-c58c7514264e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64326b5151a0472c8bf310082b2ca20d'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:49.247343 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 980/1955] 10.222.0.64 () {58 vars in 1224 bytes} [Tue Aug 2 10:00:49 2022] DELETE /identity/v3/users/9db4f7224f1c4ee48b634b3d0a9fcfb0 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:00:49.252104 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1fcdbc-3431-435d-846b-e49882a507b6 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:49.252439 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1fcdbc-3431-435d-846b-e49882a507b6 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:49.252726 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1fcdbc-3431-435d-846b-e49882a507b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:49.253944 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-a568338a-a0d8-4154-9609-c58c7514264e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64326b5151a0472c8bf310082b2ca20d'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:49.254697 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-a568338a-a0d8-4154-9609-c58c7514264e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64326b5151a0472c8bf310082b2ca20d'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:49.263426 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-a568338a-a0d8-4154-9609-c58c7514264e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64326b5151a0472c8bf310082b2ca20d'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:49.273199 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-ed1fcdbc-3431-435d-846b-e49882a507b6 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:00:49.274452 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-ed1fcdbc-3431-435d-846b-e49882a507b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:49.276931 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-a568338a-a0d8-4154-9609-c58c7514264e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64326b5151a0472c8bf310082b2ca20d'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:49.280144 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:49.284384 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 976/1956] 10.222.0.64 () {58 vars in 1224 bytes} [Tue Aug 2 10:00:49 2022] DELETE /identity/v3/users/64326b5151a0472c8bf310082b2ca20d => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:00:49.289666 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eaa491-e361-47f1-8a93-0d596b114cbd None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:49.289968 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eaa491-e361-47f1-8a93-0d596b114cbd None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:49.290254 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eaa491-e361-47f1-8a93-0d596b114cbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:49.310963 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-94eaa491-e361-47f1-8a93-0d596b114cbd None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:00:49.312215 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-94eaa491-e361-47f1-8a93-0d596b114cbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:49.341437 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 981/1957] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:00:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:00:49.350733 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1fcdbc-3431-435d-846b-e49882a507b6 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:49.352075 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-ed1fcdbc-3431-435d-846b-e49882a507b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:49.373735 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 977/1958] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:00:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:00:49.384455 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eaa491-e361-47f1-8a93-0d596b114cbd None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:49.386368 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-94eaa491-e361-47f1-8a93-0d596b114cbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:49.413531 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1fcdbc-3431-435d-846b-e49882a507b6 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:49.462378 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eaa491-e361-47f1-8a93-0d596b114cbd None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:49.467882 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea228b6-4ecf-4cf1-8649-a2cf166eda60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:49.469529 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea228b6-4ecf-4cf1-8649-a2cf166eda60 service neutron] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:49.469926 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea228b6-4ecf-4cf1-8649-a2cf166eda60 service neutron] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:49.470288 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea228b6-4ecf-4cf1-8649-a2cf166eda60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:49.484274 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea228b6-4ecf-4cf1-8649-a2cf166eda60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:49.485903 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea228b6-4ecf-4cf1-8649-a2cf166eda60 service neutron] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:49.496168 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc11517-a76e-40c8-87d4-a0e87621f418 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:49.496938 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc11517-a76e-40c8-87d4-a0e87621f418 service neutron] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:49.497292 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc11517-a76e-40c8-87d4-a0e87621f418 service neutron] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:49.497653 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc11517-a76e-40c8-87d4-a0e87621f418 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:49.498377 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 978/1959] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:00:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:00:49.509248 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc11517-a76e-40c8-87d4-a0e87621f418 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:49.510690 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc11517-a76e-40c8-87d4-a0e87621f418 service neutron] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:49.520939 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 982/1960] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:00:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:00:49.812167 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea228b6-4ecf-4cf1-8649-a2cf166eda60 service neutron] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:49.815305 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-8ea228b6-4ecf-4cf1-8649-a2cf166eda60 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:49.834425 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea228b6-4ecf-4cf1-8649-a2cf166eda60 service neutron] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:49.840720 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4446805c-8d7e-4566-b694-90d2308858ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:49.841382 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4446805c-8d7e-4566-b694-90d2308858ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:49.841670 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4446805c-8d7e-4566-b694-90d2308858ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:49.841950 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4446805c-8d7e-4566-b694-90d2308858ee admin admin] PATH_INFO: `/v3/projects/a08817dbfccc418bb57f4a1c92aca9be` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:49.846021 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4446805c-8d7e-4566-b694-90d2308858ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=a08817dbfccc418bb57f4a1c92aca9be)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:49.848305 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4446805c-8d7e-4566-b694-90d2308858ee admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:49.923415 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-4446805c-8d7e-4566-b694-90d2308858ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a08817dbfccc418bb57f4a1c92aca9be', 'request_id': 'req-4446805c-8d7e-4566-b694-90d2308858ee'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:49.928566 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:49.930964 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-4446805c-8d7e-4566-b694-90d2308858ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a08817dbfccc418bb57f4a1c92aca9be', 'request_id': 'req-4446805c-8d7e-4566-b694-90d2308858ee'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:49.939317 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-4446805c-8d7e-4566-b694-90d2308858ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a08817dbfccc418bb57f4a1c92aca9be', 'request_id': 'req-4446805c-8d7e-4566-b694-90d2308858ee'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:49.958959 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-4446805c-8d7e-4566-b694-90d2308858ee admin admin] The token cache is being invalidate because project a08817dbfccc418bb57f4a1c92aca9be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80154) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:00:49.959345 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-4446805c-8d7e-4566-b694-90d2308858ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:49.960067 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-4446805c-8d7e-4566-b694-90d2308858ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:49.962162 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 979/1961] 10.222.0.64 () {58 vars in 1233 bytes} [Tue Aug 2 10:00:49 2022] DELETE /identity/v3/projects/a08817dbfccc418bb57f4a1c92aca9be => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:00:49.965966 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc11517-a76e-40c8-87d4-a0e87621f418 service neutron] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:49.977770 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc11517-a76e-40c8-87d4-a0e87621f418 service neutron] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:50.050966 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0540f083-5c45-4746-9c6e-43d0211008e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:50.051919 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0540f083-5c45-4746-9c6e-43d0211008e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:50.052315 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0540f083-5c45-4746-9c6e-43d0211008e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:50.052633 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0540f083-5c45-4746-9c6e-43d0211008e7 admin admin] PATH_INFO: `/v3/users/50d3a33a16ad4f72ae232277739757b7` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:50.054697 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0540f083-5c45-4746-9c6e-43d0211008e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=50d3a33a16ad4f72ae232277739757b7)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:50.071015 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0540f083-5c45-4746-9c6e-43d0211008e7 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:50.090727 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:50.137062 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-0540f083-5c45-4746-9c6e-43d0211008e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50d3a33a16ad4f72ae232277739757b7'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:50.147511 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-0540f083-5c45-4746-9c6e-43d0211008e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50d3a33a16ad4f72ae232277739757b7'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:50.156901 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-0540f083-5c45-4746-9c6e-43d0211008e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50d3a33a16ad4f72ae232277739757b7'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:50.158274 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-0540f083-5c45-4746-9c6e-43d0211008e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50d3a33a16ad4f72ae232277739757b7'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:50.159317 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-0540f083-5c45-4746-9c6e-43d0211008e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50d3a33a16ad4f72ae232277739757b7'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:50.184843 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 983/1962] 10.222.0.64 () {58 vars in 1224 bytes} [Tue Aug 2 10:00:49 2022] DELETE /identity/v3/users/50d3a33a16ad4f72ae232277739757b7 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:00:50.309877 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4446805c-8d7e-4566-b694-90d2308858ee admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:50.312026 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-4446805c-8d7e-4566-b694-90d2308858ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:50.326659 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4446805c-8d7e-4566-b694-90d2308858ee admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:50.395719 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a296bd29-77c4-4f9f-9db4-b79ca11e4bcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:50.396430 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a296bd29-77c4-4f9f-9db4-b79ca11e4bcd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:50.396795 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a296bd29-77c4-4f9f-9db4-b79ca11e4bcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:50.397151 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a296bd29-77c4-4f9f-9db4-b79ca11e4bcd admin admin] PATH_INFO: `/v3/projects/8990416955af4312bc09c5212b38fd42` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:50.398412 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a296bd29-77c4-4f9f-9db4-b79ca11e4bcd admin admin] RBAC: Authorizing `identity:delete_project(project_id=8990416955af4312bc09c5212b38fd42)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:50.400528 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a296bd29-77c4-4f9f-9db4-b79ca11e4bcd admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:50.463861 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-a296bd29-77c4-4f9f-9db4-b79ca11e4bcd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8990416955af4312bc09c5212b38fd42', 'request_id': 'req-a296bd29-77c4-4f9f-9db4-b79ca11e4bcd'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:50.470940 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-a296bd29-77c4-4f9f-9db4-b79ca11e4bcd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8990416955af4312bc09c5212b38fd42', 'request_id': 'req-a296bd29-77c4-4f9f-9db4-b79ca11e4bcd'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:50.478262 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-a296bd29-77c4-4f9f-9db4-b79ca11e4bcd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8990416955af4312bc09c5212b38fd42', 'request_id': 'req-a296bd29-77c4-4f9f-9db4-b79ca11e4bcd'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:50.492998 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-a296bd29-77c4-4f9f-9db4-b79ca11e4bcd admin admin] The token cache is being invalidate because project 8990416955af4312bc09c5212b38fd42 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80154) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:00:50.493382 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-a296bd29-77c4-4f9f-9db4-b79ca11e4bcd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:50.494220 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-a296bd29-77c4-4f9f-9db4-b79ca11e4bcd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:50.495970 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 980/1963] 10.222.0.64 () {58 vars in 1233 bytes} [Tue Aug 2 10:00:50 2022] DELETE /identity/v3/projects/8990416955af4312bc09c5212b38fd42 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:00:50.500125 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0540f083-5c45-4746-9c6e-43d0211008e7 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:50.502379 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-0540f083-5c45-4746-9c6e-43d0211008e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:50.535308 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0540f083-5c45-4746-9c6e-43d0211008e7 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:50.622486 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e029c3-bf90-4fd0-a628-e4309a2189cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:50.623170 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e029c3-bf90-4fd0-a628-e4309a2189cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:50.623437 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e029c3-bf90-4fd0-a628-e4309a2189cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:50.623714 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e029c3-bf90-4fd0-a628-e4309a2189cb admin admin] PATH_INFO: `/v3/users/9be10f070ff2464da4e1d6aa7707eeaf` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:50.625048 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e029c3-bf90-4fd0-a628-e4309a2189cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=9be10f070ff2464da4e1d6aa7707eeaf)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:50.627612 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e029c3-bf90-4fd0-a628-e4309a2189cb admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:50.676346 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a296bd29-77c4-4f9f-9db4-b79ca11e4bcd admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:50.678293 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-a296bd29-77c4-4f9f-9db4-b79ca11e4bcd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:50.693429 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a296bd29-77c4-4f9f-9db4-b79ca11e4bcd admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:50.699536 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-a9e029c3-bf90-4fd0-a628-e4309a2189cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9be10f070ff2464da4e1d6aa7707eeaf'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:50.709445 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23140ce2-2926-4ca4-b98a-74a78dceef17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:50.710175 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23140ce2-2926-4ca4-b98a-74a78dceef17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:50.710474 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-a9e029c3-bf90-4fd0-a628-e4309a2189cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9be10f070ff2464da4e1d6aa7707eeaf'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:50.710614 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23140ce2-2926-4ca4-b98a-74a78dceef17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:50.710957 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23140ce2-2926-4ca4-b98a-74a78dceef17 admin admin] PATH_INFO: `/v3/projects/f8db6f6cb4a64bbf8eac38bef9033fde` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:50.713031 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23140ce2-2926-4ca4-b98a-74a78dceef17 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8db6f6cb4a64bbf8eac38bef9033fde)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:50.715742 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23140ce2-2926-4ca4-b98a-74a78dceef17 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:50.720526 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-a9e029c3-bf90-4fd0-a628-e4309a2189cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9be10f070ff2464da4e1d6aa7707eeaf'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:50.721223 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-a9e029c3-bf90-4fd0-a628-e4309a2189cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9be10f070ff2464da4e1d6aa7707eeaf'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:50.721989 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-a9e029c3-bf90-4fd0-a628-e4309a2189cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9be10f070ff2464da4e1d6aa7707eeaf'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:50.754815 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 984/1964] 10.222.0.64 () {58 vars in 1224 bytes} [Tue Aug 2 10:00:50 2022] DELETE /identity/v3/users/9be10f070ff2464da4e1d6aa7707eeaf => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:00:50.795133 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-23140ce2-2926-4ca4-b98a-74a78dceef17 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8db6f6cb4a64bbf8eac38bef9033fde', 'request_id': 'req-23140ce2-2926-4ca4-b98a-74a78dceef17'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:50.805700 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-23140ce2-2926-4ca4-b98a-74a78dceef17 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8db6f6cb4a64bbf8eac38bef9033fde', 'request_id': 'req-23140ce2-2926-4ca4-b98a-74a78dceef17'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:50.812545 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-23140ce2-2926-4ca4-b98a-74a78dceef17 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8db6f6cb4a64bbf8eac38bef9033fde', 'request_id': 'req-23140ce2-2926-4ca4-b98a-74a78dceef17'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:50.823561 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=82,keystone:INSERT=3,keystone:DELETE=15 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:50.830908 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-23140ce2-2926-4ca4-b98a-74a78dceef17 admin admin] The token cache is being invalidate because project f8db6f6cb4a64bbf8eac38bef9033fde was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80154) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:00:50.832023 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-23140ce2-2926-4ca4-b98a-74a78dceef17 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:50.833811 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-23140ce2-2926-4ca4-b98a-74a78dceef17 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:50.836122 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 981/1965] 10.222.0.64 () {58 vars in 1233 bytes} [Tue Aug 2 10:00:50 2022] DELETE /identity/v3/projects/f8db6f6cb4a64bbf8eac38bef9033fde => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:00:50.843270 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b072345f-92a0-4851-b99e-b281d5084b26 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:50.843795 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b072345f-92a0-4851-b99e-b281d5084b26 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:50.844107 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b072345f-92a0-4851-b99e-b281d5084b26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:50.873025 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-b072345f-92a0-4851-b99e-b281d5084b26 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:00:50.874557 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-b072345f-92a0-4851-b99e-b281d5084b26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:50.880451 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:50.972142 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 985/1966] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:00:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:00:50.979860 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5675d6f-f123-452e-8b2c-bdb22448661c None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:50.980221 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5675d6f-f123-452e-8b2c-bdb22448661c None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:50.980685 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5675d6f-f123-452e-8b2c-bdb22448661c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:51.003672 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-c5675d6f-f123-452e-8b2c-bdb22448661c None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:00:51.004908 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-c5675d6f-f123-452e-8b2c-bdb22448661c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:51.013589 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 982/1967] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:00:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:00:51.023029 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b072345f-92a0-4851-b99e-b281d5084b26 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:51.034495 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b072345f-92a0-4851-b99e-b281d5084b26 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:51.040207 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b36036a-25ca-408e-bd84-64895aa10022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:51.040976 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b36036a-25ca-408e-bd84-64895aa10022 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:51.043043 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b36036a-25ca-408e-bd84-64895aa10022 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:51.043043 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b36036a-25ca-408e-bd84-64895aa10022 admin admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:51.043043 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b36036a-25ca-408e-bd84-64895aa10022 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:00:51.043043 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b36036a-25ca-408e-bd84-64895aa10022 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:51.044555 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b36036a-25ca-408e-bd84-64895aa10022 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:51.057993 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 986/1968] 10.222.0.64 () {58 vars in 1153 bytes} [Tue Aug 2 10:00:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:00:51.062406 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5675d6f-f123-452e-8b2c-bdb22448661c None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:51.073466 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5675d6f-f123-452e-8b2c-bdb22448661c None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:51.079176 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad213d2-a05c-4874-9862-f163b594ce36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:51.079909 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad213d2-a05c-4874-9862-f163b594ce36 admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:51.080200 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad213d2-a05c-4874-9862-f163b594ce36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:51.080531 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad213d2-a05c-4874-9862-f163b594ce36 admin admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:51.081212 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad213d2-a05c-4874-9862-f163b594ce36 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:51.084143 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad213d2-a05c-4874-9862-f163b594ce36 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:51.120327 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 983/1969] 10.222.0.64 () {60 vars in 1153 bytes} [Tue Aug 2 10:00:51 2022] POST /identity/v3/projects => generated 373 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:00:51.124696 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b36036a-25ca-408e-bd84-64895aa10022 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:51.124833 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad213d2-a05c-4874-9862-f163b594ce36 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:51.126654 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-4ad213d2-a05c-4874-9862-f163b594ce36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:51.135148 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b36036a-25ca-408e-bd84-64895aa10022 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:51.210597 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad213d2-a05c-4874-9862-f163b594ce36 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:51.216282 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d2262e-53c0-4de9-9763-25e5091be51c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:51.216978 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d2262e-53c0-4de9-9763-25e5091be51c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:51.217285 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d2262e-53c0-4de9-9763-25e5091be51c admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:51.217555 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d2262e-53c0-4de9-9763-25e5091be51c admin admin] PATH_INFO: `/v3/projects/106f1f7c13194382acf736be0d8c9767` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:51.218763 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d2262e-53c0-4de9-9763-25e5091be51c admin admin] RBAC: Authorizing `identity:delete_project(project_id=106f1f7c13194382acf736be0d8c9767)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:51.221430 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d2262e-53c0-4de9-9763-25e5091be51c admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:51.230283 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaee8ea4-db19-4612-8891-4b6709826908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:51.230905 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaee8ea4-db19-4612-8891-4b6709826908 admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:51.231195 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaee8ea4-db19-4612-8891-4b6709826908 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:51.231470 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaee8ea4-db19-4612-8891-4b6709826908 admin admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:51.232140 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaee8ea4-db19-4612-8891-4b6709826908 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:51.234099 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaee8ea4-db19-4612-8891-4b6709826908 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:51.272270 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 987/1970] 10.222.0.64 () {60 vars in 1144 bytes} [Tue Aug 2 10:00:51 2022] POST /identity/v3/users => generated 287 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:00:51.276018 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaee8ea4-db19-4612-8891-4b6709826908 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:51.282842 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-86d2262e-53c0-4de9-9763-25e5091be51c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '106f1f7c13194382acf736be0d8c9767', 'request_id': 'req-86d2262e-53c0-4de9-9763-25e5091be51c'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:51.286728 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaee8ea4-db19-4612-8891-4b6709826908 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:51.293106 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-86d2262e-53c0-4de9-9763-25e5091be51c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '106f1f7c13194382acf736be0d8c9767', 'request_id': 'req-86d2262e-53c0-4de9-9763-25e5091be51c'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:51.300867 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-86d2262e-53c0-4de9-9763-25e5091be51c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '106f1f7c13194382acf736be0d8c9767', 'request_id': 'req-86d2262e-53c0-4de9-9763-25e5091be51c'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:51.316549 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-86d2262e-53c0-4de9-9763-25e5091be51c admin admin] The token cache is being invalidate because project 106f1f7c13194382acf736be0d8c9767 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80154) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:00:51.317258 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-86d2262e-53c0-4de9-9763-25e5091be51c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:51.317841 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-86d2262e-53c0-4de9-9763-25e5091be51c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:51.319418 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 984/1971] 10.222.0.64 () {58 vars in 1233 bytes} [Tue Aug 2 10:00:51 2022] DELETE /identity/v3/projects/106f1f7c13194382acf736be0d8c9767 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:00:51.323150 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d2262e-53c0-4de9-9763-25e5091be51c admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:51.324473 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-86d2262e-53c0-4de9-9763-25e5091be51c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:51.349792 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a92ec9-a85e-4c2e-9f1f-1a702979cfe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:51.350416 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a92ec9-a85e-4c2e-9f1f-1a702979cfe2 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:51.350701 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a92ec9-a85e-4c2e-9f1f-1a702979cfe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:51.350977 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a92ec9-a85e-4c2e-9f1f-1a702979cfe2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:51.351675 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a92ec9-a85e-4c2e-9f1f-1a702979cfe2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:51.353334 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a92ec9-a85e-4c2e-9f1f-1a702979cfe2 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:51.363348 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 988/1972] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:00:51 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:00:51.367232 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a92ec9-a85e-4c2e-9f1f-1a702979cfe2 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:51.368929 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-47a92ec9-a85e-4c2e-9f1f-1a702979cfe2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:51.380997 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a92ec9-a85e-4c2e-9f1f-1a702979cfe2 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:51.385605 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8523624c-7c13-4b23-b303-37bfd0202629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:51.386199 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8523624c-7c13-4b23-b303-37bfd0202629 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:51.386483 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8523624c-7c13-4b23-b303-37bfd0202629 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:51.386760 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8523624c-7c13-4b23-b303-37bfd0202629 admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:51.387447 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8523624c-7c13-4b23-b303-37bfd0202629 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:51.389150 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8523624c-7c13-4b23-b303-37bfd0202629 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:51.394854 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d2262e-53c0-4de9-9763-25e5091be51c admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:51.400026 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24babcb-97b0-4a8f-b472-f8004230060b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:51.400707 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24babcb-97b0-4a8f-b472-f8004230060b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:51.403888 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24babcb-97b0-4a8f-b472-f8004230060b admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:51.404008 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 989/1973] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:00:51 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:00:51.404117 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24babcb-97b0-4a8f-b472-f8004230060b admin admin] PATH_INFO: `/v3/users/c69c032f01174a51971cc8bc66cd73e0` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:51.404117 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24babcb-97b0-4a8f-b472-f8004230060b admin admin] RBAC: Authorizing `identity:delete_user(user_id=c69c032f01174a51971cc8bc66cd73e0)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:51.404706 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24babcb-97b0-4a8f-b472-f8004230060b admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:51.404924 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8523624c-7c13-4b23-b303-37bfd0202629 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:51.406444 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:51.416241 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8523624c-7c13-4b23-b303-37bfd0202629 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:51.421338 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bbc6ef-120c-4435-8608-a9e1751c2231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:51.422294 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bbc6ef-120c-4435-8608-a9e1751c2231 admin admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:51.422425 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bbc6ef-120c-4435-8608-a9e1751c2231 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:51.422693 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bbc6ef-120c-4435-8608-a9e1751c2231 admin admin] PATH_INFO: `/v3/projects/fd2e958fa0914b3f89a55d009f7a5f2b/users/9ab0dff213d34f57b1c876a3e707573f/roles/66c1074780954f8e87ed11a5c4e8c36a` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:51.432001 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:51.442084 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bbc6ef-120c-4435-8608-a9e1751c2231 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd2e958fa0914b3f89a55d009f7a5f2b, user_id=9ab0dff213d34f57b1c876a3e707573f, role_id=66c1074780954f8e87ed11a5c4e8c36a)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:51.445889 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bbc6ef-120c-4435-8608-a9e1751c2231 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:51.462516 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-a24babcb-97b0-4a8f-b472-f8004230060b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c69c032f01174a51971cc8bc66cd73e0'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:51.472282 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 990/1974] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 10:00:51 2022] PUT /identity/v3/projects/fd2e958fa0914b3f89a55d009f7a5f2b/users/9ab0dff213d34f57b1c876a3e707573f/roles/66c1074780954f8e87ed11a5c4e8c36a => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:00:51.472464 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-a24babcb-97b0-4a8f-b472-f8004230060b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c69c032f01174a51971cc8bc66cd73e0'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:51.473077 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-a24babcb-97b0-4a8f-b472-f8004230060b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c69c032f01174a51971cc8bc66cd73e0'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:51.477497 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad1de82-0678-44d7-bd1d-4f9426665226 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:51.478232 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad1de82-0678-44d7-bd1d-4f9426665226 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:51.478583 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad1de82-0678-44d7-bd1d-4f9426665226 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:51.482402 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-a24babcb-97b0-4a8f-b472-f8004230060b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c69c032f01174a51971cc8bc66cd73e0'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:51.504652 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-a24babcb-97b0-4a8f-b472-f8004230060b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c69c032f01174a51971cc8bc66cd73e0'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:51.512286 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 985/1975] 10.222.0.64 () {58 vars in 1224 bytes} [Tue Aug 2 10:00:51 2022] DELETE /identity/v3/users/c69c032f01174a51971cc8bc66cd73e0 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:00:51.518534 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-2ad1de82-0678-44d7-bd1d-4f9426665226 None None] MFA Rules not processed for user `9ab0dff213d34f57b1c876a3e707573f`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:00:51.519843 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-2ad1de82-0678-44d7-bd1d-4f9426665226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:51.602730 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 991/1976] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:00:51 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:00:51.614999 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24babcb-97b0-4a8f-b472-f8004230060b admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:51.617532 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-a24babcb-97b0-4a8f-b472-f8004230060b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:51.700129 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24babcb-97b0-4a8f-b472-f8004230060b admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:51.705664 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4ef209-6199-4897-a9ff-c0c0baffac3a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d7de62b9ee4039bdc79627ac7fb499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:51.706368 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4ef209-6199-4897-a9ff-c0c0baffac3a service cinder] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:51.706727 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4ef209-6199-4897-a9ff-c0c0baffac3a service cinder] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:51.707077 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4ef209-6199-4897-a9ff-c0c0baffac3a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:51.717130 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4ef209-6199-4897-a9ff-c0c0baffac3a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:51.718769 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4ef209-6199-4897-a9ff-c0c0baffac3a service cinder] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:51.729241 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 986/1977] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:00:51 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:00:51.817598 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad1de82-0678-44d7-bd1d-4f9426665226 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:51.819207 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-2ad1de82-0678-44d7-bd1d-4f9426665226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:51.907616 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad1de82-0678-44d7-bd1d-4f9426665226 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:51.913464 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec01ad0f-3b9b-444b-8e64-3708dff3b86c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:51.913464 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec01ad0f-3b9b-444b-8e64-3708dff3b86c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:51.913464 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec01ad0f-3b9b-444b-8e64-3708dff3b86c admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:51.913786 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec01ad0f-3b9b-444b-8e64-3708dff3b86c admin admin] PATH_INFO: `/v3/projects/a006b02764ac4027acae27f0e9022dc5` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:51.914826 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec01ad0f-3b9b-444b-8e64-3708dff3b86c admin admin] RBAC: Authorizing `identity:delete_project(project_id=a006b02764ac4027acae27f0e9022dc5)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:51.916787 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec01ad0f-3b9b-444b-8e64-3708dff3b86c admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:51.972014 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-ec01ad0f-3b9b-444b-8e64-3708dff3b86c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a006b02764ac4027acae27f0e9022dc5', 'request_id': 'req-ec01ad0f-3b9b-444b-8e64-3708dff3b86c'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:51.978495 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-ec01ad0f-3b9b-444b-8e64-3708dff3b86c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a006b02764ac4027acae27f0e9022dc5', 'request_id': 'req-ec01ad0f-3b9b-444b-8e64-3708dff3b86c'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:51.986816 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-ec01ad0f-3b9b-444b-8e64-3708dff3b86c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a006b02764ac4027acae27f0e9022dc5', 'request_id': 'req-ec01ad0f-3b9b-444b-8e64-3708dff3b86c'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:52.015505 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-ec01ad0f-3b9b-444b-8e64-3708dff3b86c admin admin] The token cache is being invalidate because project a006b02764ac4027acae27f0e9022dc5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80153) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:00:52.015746 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-ec01ad0f-3b9b-444b-8e64-3708dff3b86c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:52.020154 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-ec01ad0f-3b9b-444b-8e64-3708dff3b86c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:00:52.022373 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 992/1978] 10.222.0.64 () {58 vars in 1233 bytes} [Tue Aug 2 10:00:51 2022] DELETE /identity/v3/projects/a006b02764ac4027acae27f0e9022dc5 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:00:52.027876 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8e125a-6a63-4a5a-89e4-dc163452abc8 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:52.028173 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8e125a-6a63-4a5a-89e4-dc163452abc8 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:52.028498 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8e125a-6a63-4a5a-89e4-dc163452abc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:52.054269 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-7c8e125a-6a63-4a5a-89e4-dc163452abc8 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:00:52.055593 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-7c8e125a-6a63-4a5a-89e4-dc163452abc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:52.147060 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 987/1979] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:00:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:00:52.155866 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b163429-a315-4ba3-9ac7-368cf685780c None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:52.156154 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b163429-a315-4ba3-9ac7-368cf685780c None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:52.156496 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b163429-a315-4ba3-9ac7-368cf685780c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:52.170040 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:52.177386 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-3b163429-a315-4ba3-9ac7-368cf685780c None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:00:52.179488 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-3b163429-a315-4ba3-9ac7-368cf685780c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:52.188320 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 993/1980] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:00:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:00:52.193472 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8e125a-6a63-4a5a-89e4-dc163452abc8 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:52.204557 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8e125a-6a63-4a5a-89e4-dc163452abc8 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:52.209414 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa97954c-6947-4692-a843-2aebb0157ec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:52.210019 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa97954c-6947-4692-a843-2aebb0157ec5 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:52.210303 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa97954c-6947-4692-a843-2aebb0157ec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:52.210575 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa97954c-6947-4692-a843-2aebb0157ec5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:52.211336 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa97954c-6947-4692-a843-2aebb0157ec5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:00:52.212222 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa97954c-6947-4692-a843-2aebb0157ec5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:52.213429 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa97954c-6947-4692-a843-2aebb0157ec5 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:52.226197 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 988/1981] 10.222.0.64 () {58 vars in 1153 bytes} [Tue Aug 2 10:00:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:00:52.230648 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b163429-a315-4ba3-9ac7-368cf685780c None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:52.240747 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b163429-a315-4ba3-9ac7-368cf685780c None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:52.245691 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d177dc5-1bb7-4556-be10-e752dcd31121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:52.246301 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d177dc5-1bb7-4556-be10-e752dcd31121 admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:52.246589 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d177dc5-1bb7-4556-be10-e752dcd31121 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:52.246868 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d177dc5-1bb7-4556-be10-e752dcd31121 admin admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:52.247530 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d177dc5-1bb7-4556-be10-e752dcd31121 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:52.249463 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d177dc5-1bb7-4556-be10-e752dcd31121 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:52.276138 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 994/1982] 10.222.0.64 () {60 vars in 1153 bytes} [Tue Aug 2 10:00:52 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:00:52.280162 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa97954c-6947-4692-a843-2aebb0157ec5 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:52.290652 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa97954c-6947-4692-a843-2aebb0157ec5 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:52.296065 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:52.357682 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b53ca1f-3707-4039-8fa8-b1e4b4c0a170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:52.358379 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b53ca1f-3707-4039-8fa8-b1e4b4c0a170 admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:52.358574 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b53ca1f-3707-4039-8fa8-b1e4b4c0a170 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:52.358848 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b53ca1f-3707-4039-8fa8-b1e4b4c0a170 admin admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:52.359523 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b53ca1f-3707-4039-8fa8-b1e4b4c0a170 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:52.362211 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b53ca1f-3707-4039-8fa8-b1e4b4c0a170 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:52.388105 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 989/1983] 10.222.0.64 () {60 vars in 1144 bytes} [Tue Aug 2 10:00:52 2022] POST /identity/v3/users => generated 287 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:00:52.389597 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d177dc5-1bb7-4556-be10-e752dcd31121 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:52.400065 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d177dc5-1bb7-4556-be10-e752dcd31121 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:52.405607 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9861bd0d-ae21-41d0-8c7c-63ce6810e1eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:52.406425 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9861bd0d-ae21-41d0-8c7c-63ce6810e1eb admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:52.406637 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9861bd0d-ae21-41d0-8c7c-63ce6810e1eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:52.406976 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9861bd0d-ae21-41d0-8c7c-63ce6810e1eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:52.408284 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9861bd0d-ae21-41d0-8c7c-63ce6810e1eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:52.410094 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9861bd0d-ae21-41d0-8c7c-63ce6810e1eb admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:52.420270 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 995/1984] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:00:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:00:52.425319 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b53ca1f-3707-4039-8fa8-b1e4b4c0a170 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:52.437170 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b53ca1f-3707-4039-8fa8-b1e4b4c0a170 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:52.442220 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3f6391-3df2-4b6c-bbaa-6cec177c391b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:52.442885 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3f6391-3df2-4b6c-bbaa-6cec177c391b admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:52.443169 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3f6391-3df2-4b6c-bbaa-6cec177c391b admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:52.443456 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3f6391-3df2-4b6c-bbaa-6cec177c391b admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:52.444143 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3f6391-3df2-4b6c-bbaa-6cec177c391b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:52.445764 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3f6391-3df2-4b6c-bbaa-6cec177c391b admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:52.457571 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 990/1985] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:00:52 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:00:52.461688 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9861bd0d-ae21-41d0-8c7c-63ce6810e1eb admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:52.472915 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9861bd0d-ae21-41d0-8c7c-63ce6810e1eb admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:52.478763 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df27920e-7af2-40c5-bbd3-d6546c39e226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:52.479603 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df27920e-7af2-40c5-bbd3-d6546c39e226 admin admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:52.479951 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df27920e-7af2-40c5-bbd3-d6546c39e226 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:52.480346 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df27920e-7af2-40c5-bbd3-d6546c39e226 admin admin] PATH_INFO: `/v3/projects/1cac2783154e4570b1fbfba61b13ed92/users/ede2486b45f44ac3b0bdab2d40695bee/roles/66c1074780954f8e87ed11a5c4e8c36a` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:52.497746 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df27920e-7af2-40c5-bbd3-d6546c39e226 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cac2783154e4570b1fbfba61b13ed92, user_id=ede2486b45f44ac3b0bdab2d40695bee, role_id=66c1074780954f8e87ed11a5c4e8c36a)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:52.500580 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df27920e-7af2-40c5-bbd3-d6546c39e226 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:52.524435 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 996/1986] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 10:00:52 2022] PUT /identity/v3/projects/1cac2783154e4570b1fbfba61b13ed92/users/ede2486b45f44ac3b0bdab2d40695bee/roles/66c1074780954f8e87ed11a5c4e8c36a => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:00:52.529423 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636c77fd-740d-412b-aa58-7e4b1ec40e16 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:52.529714 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636c77fd-740d-412b-aa58-7e4b1ec40e16 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:52.529994 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636c77fd-740d-412b-aa58-7e4b1ec40e16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:52.549712 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-636c77fd-740d-412b-aa58-7e4b1ec40e16 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:00:52.550903 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-636c77fd-740d-412b-aa58-7e4b1ec40e16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:52.608909 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 991/1987] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:00:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:00:52.618422 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df27920e-7af2-40c5-bbd3-d6546c39e226 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:52.620095 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-df27920e-7af2-40c5-bbd3-d6546c39e226 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:52.677136 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df27920e-7af2-40c5-bbd3-d6546c39e226 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:52.680755 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e011caa-f4a2-4ed2-ad3f-7c52ce4ab3ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:52.681367 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e011caa-f4a2-4ed2-ad3f-7c52ce4ab3ca service neutron] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:52.681650 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e011caa-f4a2-4ed2-ad3f-7c52ce4ab3ca service neutron] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:52.681990 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e011caa-f4a2-4ed2-ad3f-7c52ce4ab3ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:52.691481 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e011caa-f4a2-4ed2-ad3f-7c52ce4ab3ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:52.692942 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e011caa-f4a2-4ed2-ad3f-7c52ce4ab3ca service neutron] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:52.702665 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 997/1988] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:00:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:00:53.526910 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae4efae-d3cc-4fd2-ac91-df280124751d None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:53.527447 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae4efae-d3cc-4fd2-ac91-df280124751d None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:53.527650 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae4efae-d3cc-4fd2-ac91-df280124751d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:53.570031 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-3ae4efae-d3cc-4fd2-ac91-df280124751d None None] MFA Rules not processed for user `ede2486b45f44ac3b0bdab2d40695bee`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:00:53.571437 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-3ae4efae-d3cc-4fd2-ac91-df280124751d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:53.665030 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 992/1989] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:00:53 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:00:53.679417 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e011caa-f4a2-4ed2-ad3f-7c52ce4ab3ca service neutron] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:00:53.681012 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-5e011caa-f4a2-4ed2-ad3f-7c52ce4ab3ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:53.751518 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e011caa-f4a2-4ed2-ad3f-7c52ce4ab3ca service neutron] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:00:53.755310 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d680ea82-a072-4b79-a7ab-aba7683ffdf3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d7de62b9ee4039bdc79627ac7fb499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:00:53.755992 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d680ea82-a072-4b79-a7ab-aba7683ffdf3 service cinder] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:53.756226 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d680ea82-a072-4b79-a7ab-aba7683ffdf3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:53.756685 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d680ea82-a072-4b79-a7ab-aba7683ffdf3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:53.778658 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d680ea82-a072-4b79-a7ab-aba7683ffdf3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:00:53.780117 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d680ea82-a072-4b79-a7ab-aba7683ffdf3 service cinder] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:00:53.790119 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 998/1990] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:00:53 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:00:54.206459 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1003f7c-4d00-426e-991b-595561dafc5f None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:54.206732 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1003f7c-4d00-426e-991b-595561dafc5f None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:54.207016 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1003f7c-4d00-426e-991b-595561dafc5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:54.225593 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-d1003f7c-4d00-426e-991b-595561dafc5f None None] MFA Rules not processed for user `9ab0dff213d34f57b1c876a3e707573f`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:00:54.226787 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-d1003f7c-4d00-426e-991b-595561dafc5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:54.238101 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:00:54.282395 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 993/1991] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:00:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:00:56.208635 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbd6320-7b00-4900-840b-142fd1c15776 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:00:56.209195 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbd6320-7b00-4900-840b-142fd1c15776 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:00:56.209195 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbd6320-7b00-4900-840b-142fd1c15776 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:00:56.229407 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-fcbd6320-7b00-4900-840b-142fd1c15776 None None] MFA Rules not processed for user `ede2486b45f44ac3b0bdab2d40695bee`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:00:56.230630 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-fcbd6320-7b00-4900-840b-142fd1c15776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:00:56.238227 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 999/1992] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:00:56 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:01:01.550125 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1003f7c-4d00-426e-991b-595561dafc5f None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:01.552457 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-d1003f7c-4d00-426e-991b-595561dafc5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:01.573878 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1003f7c-4d00-426e-991b-595561dafc5f None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:01.580417 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2df7f3-f8ad-4c44-a683-0e582ae1b8fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:01.581452 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2df7f3-f8ad-4c44-a683-0e582ae1b8fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:01.581739 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2df7f3-f8ad-4c44-a683-0e582ae1b8fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:01.582007 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2df7f3-f8ad-4c44-a683-0e582ae1b8fa admin admin] PATH_INFO: `/v3/users/de78efedd8ae4e3e8801261ba6e8e5fb` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:01.583580 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2df7f3-f8ad-4c44-a683-0e582ae1b8fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=de78efedd8ae4e3e8801261ba6e8e5fb)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:01.585825 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2df7f3-f8ad-4c44-a683-0e582ae1b8fa admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:01.668014 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-8a2df7f3-f8ad-4c44-a683-0e582ae1b8fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de78efedd8ae4e3e8801261ba6e8e5fb'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:01:01.676138 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-8a2df7f3-f8ad-4c44-a683-0e582ae1b8fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de78efedd8ae4e3e8801261ba6e8e5fb'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:01:01.676971 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-8a2df7f3-f8ad-4c44-a683-0e582ae1b8fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de78efedd8ae4e3e8801261ba6e8e5fb'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:01:01.686377 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-8a2df7f3-f8ad-4c44-a683-0e582ae1b8fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de78efedd8ae4e3e8801261ba6e8e5fb'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:01:01.699376 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-8a2df7f3-f8ad-4c44-a683-0e582ae1b8fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de78efedd8ae4e3e8801261ba6e8e5fb'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:01:01.706017 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 994/1993] 10.222.0.64 () {58 vars in 1224 bytes} [Tue Aug 2 10:01:01 2022] DELETE /identity/v3/users/de78efedd8ae4e3e8801261ba6e8e5fb => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:01:01.710580 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d114bdc2-626b-405d-91dd-bd3cdfde77f2 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:01.711087 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d114bdc2-626b-405d-91dd-bd3cdfde77f2 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:01.711257 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d114bdc2-626b-405d-91dd-bd3cdfde77f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:01.730572 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-d114bdc2-626b-405d-91dd-bd3cdfde77f2 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:01.731831 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-d114bdc2-626b-405d-91dd-bd3cdfde77f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:01.777501 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:01.803374 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1000/1994] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:01:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:01:01.827862 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2df7f3-f8ad-4c44-a683-0e582ae1b8fa admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:01.831116 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-8a2df7f3-f8ad-4c44-a683-0e582ae1b8fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:01.912499 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2df7f3-f8ad-4c44-a683-0e582ae1b8fa admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:01.916509 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02d40e7-6966-4227-b0b6-a0bf958877ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:01.917905 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02d40e7-6966-4227-b0b6-a0bf958877ce service neutron] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:01.918342 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02d40e7-6966-4227-b0b6-a0bf958877ce service neutron] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:01.918587 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02d40e7-6966-4227-b0b6-a0bf958877ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:01.933403 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02d40e7-6966-4227-b0b6-a0bf958877ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:01.934837 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02d40e7-6966-4227-b0b6-a0bf958877ce service neutron] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:01.945242 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 995/1995] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:01:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:01:02.216040 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d114bdc2-626b-405d-91dd-bd3cdfde77f2 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:02.218051 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-d114bdc2-626b-405d-91dd-bd3cdfde77f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:02.233151 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d114bdc2-626b-405d-91dd-bd3cdfde77f2 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:02.237816 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffdd8c7-182e-49fd-a2e6-767537488363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:02.238416 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffdd8c7-182e-49fd-a2e6-767537488363 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:02.238697 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffdd8c7-182e-49fd-a2e6-767537488363 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:02.238983 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffdd8c7-182e-49fd-a2e6-767537488363 admin admin] PATH_INFO: `/v3/projects/f15edeeaae474d4ab94e9dcb652da720` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:02.240237 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffdd8c7-182e-49fd-a2e6-767537488363 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f15edeeaae474d4ab94e9dcb652da720)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:02.242155 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffdd8c7-182e-49fd-a2e6-767537488363 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:02.302403 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-2ffdd8c7-182e-49fd-a2e6-767537488363 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f15edeeaae474d4ab94e9dcb652da720', 'request_id': 'req-2ffdd8c7-182e-49fd-a2e6-767537488363'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:01:02.307941 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-2ffdd8c7-182e-49fd-a2e6-767537488363 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f15edeeaae474d4ab94e9dcb652da720', 'request_id': 'req-2ffdd8c7-182e-49fd-a2e6-767537488363'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:01:02.314944 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-2ffdd8c7-182e-49fd-a2e6-767537488363 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f15edeeaae474d4ab94e9dcb652da720', 'request_id': 'req-2ffdd8c7-182e-49fd-a2e6-767537488363'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:01:02.328118 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-2ffdd8c7-182e-49fd-a2e6-767537488363 admin admin] The token cache is being invalidate because project f15edeeaae474d4ab94e9dcb652da720 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80153) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:01:02.328434 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-2ffdd8c7-182e-49fd-a2e6-767537488363 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:01:02.329057 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-2ffdd8c7-182e-49fd-a2e6-767537488363 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:01:02.331628 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1001/1996] 10.222.0.64 () {58 vars in 1233 bytes} [Tue Aug 2 10:01:02 2022] DELETE /identity/v3/projects/f15edeeaae474d4ab94e9dcb652da720 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:01:02.336838 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e067cc-2ee7-463d-8c12-6b17df9efa24 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:02.337385 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e067cc-2ee7-463d-8c12-6b17df9efa24 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:02.338015 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e067cc-2ee7-463d-8c12-6b17df9efa24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:02.361268 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-81e067cc-2ee7-463d-8c12-6b17df9efa24 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:02.362515 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-81e067cc-2ee7-463d-8c12-6b17df9efa24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:02.407183 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:02.443731 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 996/1997] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:01:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:01:02.448540 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120819fa-ba26-4afb-8ec4-b63867a050bb None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:02.448914 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120819fa-ba26-4afb-8ec4-b63867a050bb None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:02.449186 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120819fa-ba26-4afb-8ec4-b63867a050bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:02.469966 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-120819fa-ba26-4afb-8ec4-b63867a050bb None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:02.471111 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-120819fa-ba26-4afb-8ec4-b63867a050bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:02.484411 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1002/1998] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:01:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:01:02.485516 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e067cc-2ee7-463d-8c12-6b17df9efa24 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:02.495589 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e067cc-2ee7-463d-8c12-6b17df9efa24 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:02.500275 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499ea5e1-224b-4416-adc2-e78eeb261991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:02.500917 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499ea5e1-224b-4416-adc2-e78eeb261991 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:02.501208 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499ea5e1-224b-4416-adc2-e78eeb261991 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:02.502001 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499ea5e1-224b-4416-adc2-e78eeb261991 admin admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:02.502718 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499ea5e1-224b-4416-adc2-e78eeb261991 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:02.503029 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499ea5e1-224b-4416-adc2-e78eeb261991 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:02.504654 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499ea5e1-224b-4416-adc2-e78eeb261991 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:02.516418 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 997/1999] 10.222.0.64 () {58 vars in 1153 bytes} [Tue Aug 2 10:01:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:02.521309 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120819fa-ba26-4afb-8ec4-b63867a050bb None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:02.530397 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120819fa-ba26-4afb-8ec4-b63867a050bb None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:02.535538 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9289f334-c071-4884-9568-324472e6d859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:02.536153 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9289f334-c071-4884-9568-324472e6d859 admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:02.536476 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9289f334-c071-4884-9568-324472e6d859 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:02.537227 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9289f334-c071-4884-9568-324472e6d859 admin admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:02.537433 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9289f334-c071-4884-9568-324472e6d859 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:02.539330 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9289f334-c071-4884-9568-324472e6d859 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:02.575310 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1003/2000] 10.222.0.64 () {60 vars in 1153 bytes} [Tue Aug 2 10:01:02 2022] POST /identity/v3/projects => generated 375 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:01:02.576862 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499ea5e1-224b-4416-adc2-e78eeb261991 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:02.609274 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499ea5e1-224b-4416-adc2-e78eeb261991 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:02.692152 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f93747e-e057-453d-a966-f1e216e77600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:02.692821 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f93747e-e057-453d-a966-f1e216e77600 admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:02.693113 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f93747e-e057-453d-a966-f1e216e77600 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:02.693390 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f93747e-e057-453d-a966-f1e216e77600 admin admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:02.694074 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f93747e-e057-453d-a966-f1e216e77600 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:02.695979 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f93747e-e057-453d-a966-f1e216e77600 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:02.724229 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 998/2001] 10.222.0.64 () {60 vars in 1144 bytes} [Tue Aug 2 10:01:02 2022] POST /identity/v3/users => generated 288 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:01:02.728714 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9289f334-c071-4884-9568-324472e6d859 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:02.740362 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9289f334-c071-4884-9568-324472e6d859 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:02.745590 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b1c87e-46c2-46be-ac59-efe24b40ef58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:02.746398 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b1c87e-46c2-46be-ac59-efe24b40ef58 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:02.746685 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b1c87e-46c2-46be-ac59-efe24b40ef58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:02.746958 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b1c87e-46c2-46be-ac59-efe24b40ef58 admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:02.747678 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b1c87e-46c2-46be-ac59-efe24b40ef58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:02.749308 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b1c87e-46c2-46be-ac59-efe24b40ef58 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:02.761889 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1004/2002] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:01:02 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:01:02.766825 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f93747e-e057-453d-a966-f1e216e77600 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:02.780688 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f93747e-e057-453d-a966-f1e216e77600 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:02.785915 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b093a9-9009-4664-89ed-f01764be4633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:02.786534 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b093a9-9009-4664-89ed-f01764be4633 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:02.786864 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b093a9-9009-4664-89ed-f01764be4633 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:02.787240 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b093a9-9009-4664-89ed-f01764be4633 admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:02.788123 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b093a9-9009-4664-89ed-f01764be4633 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:02.789986 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b093a9-9009-4664-89ed-f01764be4633 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:02.803296 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 999/2003] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:01:02 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:01:02.807372 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b1c87e-46c2-46be-ac59-efe24b40ef58 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:02.819983 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b1c87e-46c2-46be-ac59-efe24b40ef58 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:02.825250 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e83520-b1f8-476a-a620-8c481f405c57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:02.825955 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e83520-b1f8-476a-a620-8c481f405c57 admin admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:02.826207 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e83520-b1f8-476a-a620-8c481f405c57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:02.826750 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e83520-b1f8-476a-a620-8c481f405c57 admin admin] PATH_INFO: `/v3/projects/cde4b642e0274ba888fcee5647d2e27a/users/39656931b2ff4c31b403bb1121f40ecf/roles/66c1074780954f8e87ed11a5c4e8c36a` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:02.852264 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e83520-b1f8-476a-a620-8c481f405c57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cde4b642e0274ba888fcee5647d2e27a, user_id=39656931b2ff4c31b403bb1121f40ecf, role_id=66c1074780954f8e87ed11a5c4e8c36a)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:02.854998 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e83520-b1f8-476a-a620-8c481f405c57 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:02.870958 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1005/2004] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 10:01:02 2022] PUT /identity/v3/projects/cde4b642e0274ba888fcee5647d2e27a/users/39656931b2ff4c31b403bb1121f40ecf/roles/66c1074780954f8e87ed11a5c4e8c36a => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:01:02.881566 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce2d753-4026-46e0-b375-9570a599fdd1 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:02.881847 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce2d753-4026-46e0-b375-9570a599fdd1 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:02.882100 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce2d753-4026-46e0-b375-9570a599fdd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:02.917678 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-cce2d753-4026-46e0-b375-9570a599fdd1 None None] MFA Rules not processed for user `39656931b2ff4c31b403bb1121f40ecf`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:02.919013 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-cce2d753-4026-46e0-b375-9570a599fdd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:03.017002 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1000/2005] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:01:02 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:01:03.018560 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e83520-b1f8-476a-a620-8c481f405c57 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:03.093239 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e83520-b1f8-476a-a620-8c481f405c57 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:03.098627 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d57d0c-0118-4172-823b-8162ebef9bef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:03.099255 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d57d0c-0118-4172-823b-8162ebef9bef admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:03.099540 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d57d0c-0118-4172-823b-8162ebef9bef admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:03.099819 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d57d0c-0118-4172-823b-8162ebef9bef admin admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:03.100560 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d57d0c-0118-4172-823b-8162ebef9bef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:03.102519 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d57d0c-0118-4172-823b-8162ebef9bef admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:03.108075 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:03.137497 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1006/2006] 10.222.0.64 () {60 vars in 1153 bytes} [Tue Aug 2 10:01:03 2022] POST /identity/v3/projects => generated 377 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:01:03.141535 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce2d753-4026-46e0-b375-9570a599fdd1 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:03.152312 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce2d753-4026-46e0-b375-9570a599fdd1 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:03.183781 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:03.210868 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643a7908-9644-4c41-883b-57d5efa5c704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:03.211489 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643a7908-9644-4c41-883b-57d5efa5c704 admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:03.211775 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643a7908-9644-4c41-883b-57d5efa5c704 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:03.212182 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643a7908-9644-4c41-883b-57d5efa5c704 admin admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:03.212917 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643a7908-9644-4c41-883b-57d5efa5c704 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:03.214849 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643a7908-9644-4c41-883b-57d5efa5c704 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:03.235257 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1001/2007] 10.222.0.64 () {60 vars in 1144 bytes} [Tue Aug 2 10:01:03 2022] POST /identity/v3/users => generated 289 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:01:03.239867 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d57d0c-0118-4172-823b-8162ebef9bef admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:03.250082 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d57d0c-0118-4172-823b-8162ebef9bef admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:03.254992 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2d089d-0dd3-4a41-af2c-2c12d2ac12f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:03.255543 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2d089d-0dd3-4a41-af2c-2c12d2ac12f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:03.255830 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2d089d-0dd3-4a41-af2c-2c12d2ac12f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:03.256109 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2d089d-0dd3-4a41-af2c-2c12d2ac12f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:03.256838 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2d089d-0dd3-4a41-af2c-2c12d2ac12f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:03.258432 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2d089d-0dd3-4a41-af2c-2c12d2ac12f2 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:03.269284 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1007/2008] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:01:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:01:03.273232 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643a7908-9644-4c41-883b-57d5efa5c704 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:03.283799 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643a7908-9644-4c41-883b-57d5efa5c704 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:03.288470 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8699116d-cd3a-4c42-b852-1868440e711b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:03.289072 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8699116d-cd3a-4c42-b852-1868440e711b admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:03.289359 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8699116d-cd3a-4c42-b852-1868440e711b admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:03.289634 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8699116d-cd3a-4c42-b852-1868440e711b admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:03.290330 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8699116d-cd3a-4c42-b852-1868440e711b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:03.291903 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8699116d-cd3a-4c42-b852-1868440e711b admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:03.301756 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1002/2009] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:01:03 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:01:03.305904 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2d089d-0dd3-4a41-af2c-2c12d2ac12f2 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:03.315752 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2d089d-0dd3-4a41-af2c-2c12d2ac12f2 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:03.320417 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ece70c-76a4-4ef3-9a85-d908b8df8964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:03.321069 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ece70c-76a4-4ef3-9a85-d908b8df8964 admin admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:03.321356 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ece70c-76a4-4ef3-9a85-d908b8df8964 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:03.321635 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ece70c-76a4-4ef3-9a85-d908b8df8964 admin admin] PATH_INFO: `/v3/projects/de2d39a826844a7585f03aa91f63a0ed/users/5feec018cae6453fb0fcca0e000e7544/roles/66c1074780954f8e87ed11a5c4e8c36a` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:03.338612 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ece70c-76a4-4ef3-9a85-d908b8df8964 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de2d39a826844a7585f03aa91f63a0ed, user_id=5feec018cae6453fb0fcca0e000e7544, role_id=66c1074780954f8e87ed11a5c4e8c36a)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:03.341330 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ece70c-76a4-4ef3-9a85-d908b8df8964 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:03.354753 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1008/2010] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 10:01:03 2022] PUT /identity/v3/projects/de2d39a826844a7585f03aa91f63a0ed/users/5feec018cae6453fb0fcca0e000e7544/roles/66c1074780954f8e87ed11a5c4e8c36a => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:01:03.365291 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de09c15-8d17-4454-aeef-680f696f8a45 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:03.365600 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de09c15-8d17-4454-aeef-680f696f8a45 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:03.365857 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de09c15-8d17-4454-aeef-680f696f8a45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:03.393441 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-7de09c15-8d17-4454-aeef-680f696f8a45 None None] MFA Rules not processed for user `5feec018cae6453fb0fcca0e000e7544`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:03.394645 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-7de09c15-8d17-4454-aeef-680f696f8a45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:03.458844 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1003/2011] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:01:03 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:01:03.463275 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ece70c-76a4-4ef3-9a85-d908b8df8964 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:03.524155 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ece70c-76a4-4ef3-9a85-d908b8df8964 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:03.528846 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd130d1c-a30d-4613-b520-8d0ab8b07ba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:03.529450 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd130d1c-a30d-4613-b520-8d0ab8b07ba8 admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:03.529738 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd130d1c-a30d-4613-b520-8d0ab8b07ba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:03.530016 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd130d1c-a30d-4613-b520-8d0ab8b07ba8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:03.530693 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd130d1c-a30d-4613-b520-8d0ab8b07ba8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:03.532644 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd130d1c-a30d-4613-b520-8d0ab8b07ba8 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:03.559208 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1009/2012] 10.222.0.64 () {60 vars in 1153 bytes} [Tue Aug 2 10:01:03 2022] POST /identity/v3/projects => generated 377 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:01:03.563339 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de09c15-8d17-4454-aeef-680f696f8a45 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:03.574176 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de09c15-8d17-4454-aeef-680f696f8a45 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:03.642872 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4369b990-2bf1-418a-b618-3ed1d167dcaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:03.643549 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4369b990-2bf1-418a-b618-3ed1d167dcaa admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:03.643874 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4369b990-2bf1-418a-b618-3ed1d167dcaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:03.644284 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4369b990-2bf1-418a-b618-3ed1d167dcaa admin admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:03.645418 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4369b990-2bf1-418a-b618-3ed1d167dcaa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:03.647411 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4369b990-2bf1-418a-b618-3ed1d167dcaa admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:03.673783 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1004/2013] 10.222.0.64 () {60 vars in 1144 bytes} [Tue Aug 2 10:01:03 2022] POST /identity/v3/users => generated 295 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:01:03.678162 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd130d1c-a30d-4613-b520-8d0ab8b07ba8 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:03.690097 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd130d1c-a30d-4613-b520-8d0ab8b07ba8 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:03.695168 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9b3113-0eaa-4549-ae10-b3526fbbcb1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:03.695830 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9b3113-0eaa-4549-ae10-b3526fbbcb1b admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:03.696119 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9b3113-0eaa-4549-ae10-b3526fbbcb1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:03.696442 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9b3113-0eaa-4549-ae10-b3526fbbcb1b admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:03.697155 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9b3113-0eaa-4549-ae10-b3526fbbcb1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:03.698873 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9b3113-0eaa-4549-ae10-b3526fbbcb1b admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:03.711248 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1010/2014] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:01:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:01:03.715831 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4369b990-2bf1-418a-b618-3ed1d167dcaa admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:03.726423 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4369b990-2bf1-418a-b618-3ed1d167dcaa admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:03.731448 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8100069a-b7e3-4c16-ab5b-9d6f1518642e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:03.732890 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8100069a-b7e3-4c16-ab5b-9d6f1518642e admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:03.733361 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8100069a-b7e3-4c16-ab5b-9d6f1518642e admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:03.733650 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8100069a-b7e3-4c16-ab5b-9d6f1518642e admin admin] PATH_INFO: `/v3/projects/5f234bc78f854284a5a21f68297c16a2/users/00670dad22ad46efbb266d8c1c7e427b/roles/1000cf318fa14915b8091599eaead5de` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:03.749579 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8100069a-b7e3-4c16-ab5b-9d6f1518642e admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f234bc78f854284a5a21f68297c16a2, user_id=00670dad22ad46efbb266d8c1c7e427b, role_id=1000cf318fa14915b8091599eaead5de)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:03.752986 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8100069a-b7e3-4c16-ab5b-9d6f1518642e admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:03.768370 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1005/2015] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 10:01:03 2022] PUT /identity/v3/projects/5f234bc78f854284a5a21f68297c16a2/users/00670dad22ad46efbb266d8c1c7e427b/roles/1000cf318fa14915b8091599eaead5de => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:01:03.773412 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9b3113-0eaa-4549-ae10-b3526fbbcb1b admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:03.783876 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9b3113-0eaa-4549-ae10-b3526fbbcb1b admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:03.875066 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee91a77-3a75-48cb-ae33-6089e1abe707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:03.876381 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee91a77-3a75-48cb-ae33-6089e1abe707 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:03.876381 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee91a77-3a75-48cb-ae33-6089e1abe707 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:03.876381 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee91a77-3a75-48cb-ae33-6089e1abe707 admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:03.877778 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee91a77-3a75-48cb-ae33-6089e1abe707 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:03.879491 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee91a77-3a75-48cb-ae33-6089e1abe707 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:03.898438 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1011/2016] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:01:03 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:01:03.898964 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8100069a-b7e3-4c16-ab5b-9d6f1518642e admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:03.913359 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8100069a-b7e3-4c16-ab5b-9d6f1518642e admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:03.918439 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6169b60e-e57b-4b36-a111-7a824b5caa10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:03.919223 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6169b60e-e57b-4b36-a111-7a824b5caa10 admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:03.919584 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6169b60e-e57b-4b36-a111-7a824b5caa10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:03.919937 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6169b60e-e57b-4b36-a111-7a824b5caa10 admin admin] PATH_INFO: `/v3/projects/5f234bc78f854284a5a21f68297c16a2/users/00670dad22ad46efbb266d8c1c7e427b/roles/1000cf318fa14915b8091599eaead5de` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:03.922092 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6169b60e-e57b-4b36-a111-7a824b5caa10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f234bc78f854284a5a21f68297c16a2, user_id=00670dad22ad46efbb266d8c1c7e427b, role_id=1000cf318fa14915b8091599eaead5de)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:03.924918 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6169b60e-e57b-4b36-a111-7a824b5caa10 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:03.927474 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:03.953009 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1006/2017] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 10:01:03 2022] PUT /identity/v3/projects/5f234bc78f854284a5a21f68297c16a2/users/00670dad22ad46efbb266d8c1c7e427b/roles/1000cf318fa14915b8091599eaead5de => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:01:03.965828 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257ee7a1-be5e-491d-86ca-c288695fe6f8 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:03.966214 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257ee7a1-be5e-491d-86ca-c288695fe6f8 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:03.966960 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257ee7a1-be5e-491d-86ca-c288695fe6f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:04.007400 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-257ee7a1-be5e-491d-86ca-c288695fe6f8 None None] MFA Rules not processed for user `00670dad22ad46efbb266d8c1c7e427b`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:04.008904 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-257ee7a1-be5e-491d-86ca-c288695fe6f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:04.019733 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:04.136585 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1012/2018] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:01:03 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:01:04.150364 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6169b60e-e57b-4b36-a111-7a824b5caa10 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:04.152120 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-6169b60e-e57b-4b36-a111-7a824b5caa10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:04.225029 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6169b60e-e57b-4b36-a111-7a824b5caa10 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:04.229087 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fac1dc-19c9-4fee-aeec-12052b074c0b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d7de62b9ee4039bdc79627ac7fb499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:04.229715 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fac1dc-19c9-4fee-aeec-12052b074c0b service cinder] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:04.229999 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fac1dc-19c9-4fee-aeec-12052b074c0b service cinder] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:04.230273 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fac1dc-19c9-4fee-aeec-12052b074c0b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:04.299705 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fac1dc-19c9-4fee-aeec-12052b074c0b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:04.301466 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fac1dc-19c9-4fee-aeec-12052b074c0b service cinder] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:04.313517 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1007/2019] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:01:04 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:01:05.846367 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257ee7a1-be5e-491d-86ca-c288695fe6f8 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:05.847877 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-257ee7a1-be5e-491d-86ca-c288695fe6f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:05.860042 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257ee7a1-be5e-491d-86ca-c288695fe6f8 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:05.863680 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ba8bfe-c6fe-4387-b981-49cd2c4bca0e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d7de62b9ee4039bdc79627ac7fb499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:05.864330 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ba8bfe-c6fe-4387-b981-49cd2c4bca0e service cinder] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:05.864626 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ba8bfe-c6fe-4387-b981-49cd2c4bca0e service cinder] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:05.864904 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ba8bfe-c6fe-4387-b981-49cd2c4bca0e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:05.923498 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ba8bfe-c6fe-4387-b981-49cd2c4bca0e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:05.924902 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ba8bfe-c6fe-4387-b981-49cd2c4bca0e service cinder] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:05.933644 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1013/2020] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:01:05 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:01:06.671932 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fac1dc-19c9-4fee-aeec-12052b074c0b service cinder] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:06.682744 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fac1dc-19c9-4fee-aeec-12052b074c0b service cinder] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:06.696658 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dbb7d8-5c20-4dbb-9bac-816ffed921c4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d7de62b9ee4039bdc79627ac7fb499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:06.698480 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dbb7d8-5c20-4dbb-9bac-816ffed921c4 service cinder] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:06.699097 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dbb7d8-5c20-4dbb-9bac-816ffed921c4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:06.699597 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dbb7d8-5c20-4dbb-9bac-816ffed921c4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:06.712710 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dbb7d8-5c20-4dbb-9bac-816ffed921c4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:06.714817 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dbb7d8-5c20-4dbb-9bac-816ffed921c4 service cinder] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:06.727228 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1008/2021] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:01:06 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:01:11.038932 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ba8bfe-c6fe-4387-b981-49cd2c4bca0e service cinder] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:11.139603 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ba8bfe-c6fe-4387-b981-49cd2c4bca0e service cinder] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:11.146271 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3788944-acf1-4d0c-8255-2b2347f0f69b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:11.147151 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3788944-acf1-4d0c-8255-2b2347f0f69b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:11.147596 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3788944-acf1-4d0c-8255-2b2347f0f69b admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:11.147894 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3788944-acf1-4d0c-8255-2b2347f0f69b admin admin] PATH_INFO: `/v3/users/39656931b2ff4c31b403bb1121f40ecf` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:11.149598 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3788944-acf1-4d0c-8255-2b2347f0f69b admin admin] RBAC: Authorizing `identity:delete_user(user_id=39656931b2ff4c31b403bb1121f40ecf)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:11.152335 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3788944-acf1-4d0c-8255-2b2347f0f69b admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:11.235609 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-b3788944-acf1-4d0c-8255-2b2347f0f69b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39656931b2ff4c31b403bb1121f40ecf'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:01:11.237145 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:11.246321 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-b3788944-acf1-4d0c-8255-2b2347f0f69b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39656931b2ff4c31b403bb1121f40ecf'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:01:11.260026 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-b3788944-acf1-4d0c-8255-2b2347f0f69b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39656931b2ff4c31b403bb1121f40ecf'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:01:11.261210 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-b3788944-acf1-4d0c-8255-2b2347f0f69b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39656931b2ff4c31b403bb1121f40ecf'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:01:11.262117 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-b3788944-acf1-4d0c-8255-2b2347f0f69b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39656931b2ff4c31b403bb1121f40ecf'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:01:11.291900 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1014/2022] 10.222.0.64 () {58 vars in 1224 bytes} [Tue Aug 2 10:01:11 2022] DELETE /identity/v3/users/39656931b2ff4c31b403bb1121f40ecf => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:01:11.297071 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025b5941-5bc1-489a-98cd-9b946f87bae8 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:11.297620 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025b5941-5bc1-489a-98cd-9b946f87bae8 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:11.297729 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025b5941-5bc1-489a-98cd-9b946f87bae8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:11.328109 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-025b5941-5bc1-489a-98cd-9b946f87bae8 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:11.329854 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-025b5941-5bc1-489a-98cd-9b946f87bae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:11.418024 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1009/2023] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:01:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:01:11.425447 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3788944-acf1-4d0c-8255-2b2347f0f69b admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:11.426963 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-b3788944-acf1-4d0c-8255-2b2347f0f69b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:11.503288 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3788944-acf1-4d0c-8255-2b2347f0f69b admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:11.507384 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef960a11-73cd-4664-ab89-2981168d3919 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:11.508069 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef960a11-73cd-4664-ab89-2981168d3919 service neutron] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:11.508390 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef960a11-73cd-4664-ab89-2981168d3919 service neutron] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:11.508736 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef960a11-73cd-4664-ab89-2981168d3919 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:11.523387 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef960a11-73cd-4664-ab89-2981168d3919 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:11.525213 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef960a11-73cd-4664-ab89-2981168d3919 service neutron] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:11.537689 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1015/2024] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:01:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:01:12.604381 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025b5941-5bc1-489a-98cd-9b946f87bae8 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:12.605912 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-025b5941-5bc1-489a-98cd-9b946f87bae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:12.618631 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025b5941-5bc1-489a-98cd-9b946f87bae8 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:12.624943 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010da6a4-e0b3-4cd9-87cc-e6392fe723e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:12.630952 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010da6a4-e0b3-4cd9-87cc-e6392fe723e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:12.631761 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010da6a4-e0b3-4cd9-87cc-e6392fe723e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:12.632072 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010da6a4-e0b3-4cd9-87cc-e6392fe723e6 admin admin] PATH_INFO: `/v3/projects/cde4b642e0274ba888fcee5647d2e27a` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:12.633439 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010da6a4-e0b3-4cd9-87cc-e6392fe723e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cde4b642e0274ba888fcee5647d2e27a)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:12.635437 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010da6a4-e0b3-4cd9-87cc-e6392fe723e6 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:12.682894 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:INSERT=1,keystone:SELECT=97,keystone:DELETE=2 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:12.701020 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-010da6a4-e0b3-4cd9-87cc-e6392fe723e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cde4b642e0274ba888fcee5647d2e27a', 'request_id': 'req-010da6a4-e0b3-4cd9-87cc-e6392fe723e6'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:01:12.708648 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-010da6a4-e0b3-4cd9-87cc-e6392fe723e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cde4b642e0274ba888fcee5647d2e27a', 'request_id': 'req-010da6a4-e0b3-4cd9-87cc-e6392fe723e6'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:01:12.715226 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-010da6a4-e0b3-4cd9-87cc-e6392fe723e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cde4b642e0274ba888fcee5647d2e27a', 'request_id': 'req-010da6a4-e0b3-4cd9-87cc-e6392fe723e6'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:01:12.730167 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-010da6a4-e0b3-4cd9-87cc-e6392fe723e6 admin admin] The token cache is being invalidate because project cde4b642e0274ba888fcee5647d2e27a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80154) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:01:12.730462 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-010da6a4-e0b3-4cd9-87cc-e6392fe723e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:01:12.731094 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-010da6a4-e0b3-4cd9-87cc-e6392fe723e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:01:12.736332 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1010/2025] 10.222.0.64 () {58 vars in 1233 bytes} [Tue Aug 2 10:01:12 2022] DELETE /identity/v3/projects/cde4b642e0274ba888fcee5647d2e27a => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:01:12.738635 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef960a11-73cd-4664-ab89-2981168d3919 service neutron] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:12.755367 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef960a11-73cd-4664-ab89-2981168d3919 service neutron] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:12.817650 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed64b37-a2b9-41df-a538-db3d06cff0ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:12.818280 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed64b37-a2b9-41df-a538-db3d06cff0ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:12.818563 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed64b37-a2b9-41df-a538-db3d06cff0ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:12.818840 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed64b37-a2b9-41df-a538-db3d06cff0ad admin admin] PATH_INFO: `/v3/users/5feec018cae6453fb0fcca0e000e7544` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:12.820223 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed64b37-a2b9-41df-a538-db3d06cff0ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=5feec018cae6453fb0fcca0e000e7544)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:12.823274 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed64b37-a2b9-41df-a538-db3d06cff0ad admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:12.872005 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-7ed64b37-a2b9-41df-a538-db3d06cff0ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5feec018cae6453fb0fcca0e000e7544'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:01:12.878923 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-7ed64b37-a2b9-41df-a538-db3d06cff0ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5feec018cae6453fb0fcca0e000e7544'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:01:12.886327 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-7ed64b37-a2b9-41df-a538-db3d06cff0ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5feec018cae6453fb0fcca0e000e7544'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:01:12.886991 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-7ed64b37-a2b9-41df-a538-db3d06cff0ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5feec018cae6453fb0fcca0e000e7544'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:01:12.887730 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-7ed64b37-a2b9-41df-a538-db3d06cff0ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5feec018cae6453fb0fcca0e000e7544'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:01:12.894885 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:12.909947 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1016/2026] 10.222.0.64 () {58 vars in 1224 bytes} [Tue Aug 2 10:01:12 2022] DELETE /identity/v3/users/5feec018cae6453fb0fcca0e000e7544 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:01:13.252231 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010da6a4-e0b3-4cd9-87cc-e6392fe723e6 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:13.253753 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-010da6a4-e0b3-4cd9-87cc-e6392fe723e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:13.273362 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010da6a4-e0b3-4cd9-87cc-e6392fe723e6 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:13.376734 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b996399-55d6-4d3e-8cb1-90a8bbd831a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:13.377482 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b996399-55d6-4d3e-8cb1-90a8bbd831a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:13.377839 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b996399-55d6-4d3e-8cb1-90a8bbd831a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:13.378123 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b996399-55d6-4d3e-8cb1-90a8bbd831a9 admin admin] PATH_INFO: `/v3/projects/de2d39a826844a7585f03aa91f63a0ed` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:13.379337 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b996399-55d6-4d3e-8cb1-90a8bbd831a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=de2d39a826844a7585f03aa91f63a0ed)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:13.381362 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b996399-55d6-4d3e-8cb1-90a8bbd831a9 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:13.468508 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-3b996399-55d6-4d3e-8cb1-90a8bbd831a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de2d39a826844a7585f03aa91f63a0ed', 'request_id': 'req-3b996399-55d6-4d3e-8cb1-90a8bbd831a9'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:01:13.476778 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-3b996399-55d6-4d3e-8cb1-90a8bbd831a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de2d39a826844a7585f03aa91f63a0ed', 'request_id': 'req-3b996399-55d6-4d3e-8cb1-90a8bbd831a9'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:01:13.488398 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-3b996399-55d6-4d3e-8cb1-90a8bbd831a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de2d39a826844a7585f03aa91f63a0ed', 'request_id': 'req-3b996399-55d6-4d3e-8cb1-90a8bbd831a9'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:01:13.509664 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-3b996399-55d6-4d3e-8cb1-90a8bbd831a9 admin admin] The token cache is being invalidate because project de2d39a826844a7585f03aa91f63a0ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80154) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:01:13.510970 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-3b996399-55d6-4d3e-8cb1-90a8bbd831a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:01:13.512238 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-3b996399-55d6-4d3e-8cb1-90a8bbd831a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:01:13.515856 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1011/2027] 10.222.0.64 () {58 vars in 1233 bytes} [Tue Aug 2 10:01:13 2022] DELETE /identity/v3/projects/de2d39a826844a7585f03aa91f63a0ed => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:01:13.520610 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed64b37-a2b9-41df-a538-db3d06cff0ad admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:13.522148 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-7ed64b37-a2b9-41df-a538-db3d06cff0ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:13.541334 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed64b37-a2b9-41df-a538-db3d06cff0ad admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:13.623360 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa0ba61-6f8c-4938-acdb-887a2cb49162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:13.624516 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa0ba61-6f8c-4938-acdb-887a2cb49162 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:13.625024 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa0ba61-6f8c-4938-acdb-887a2cb49162 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:13.625553 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa0ba61-6f8c-4938-acdb-887a2cb49162 admin admin] PATH_INFO: `/v3/users/00670dad22ad46efbb266d8c1c7e427b` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:13.627961 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa0ba61-6f8c-4938-acdb-887a2cb49162 admin admin] RBAC: Authorizing `identity:delete_user(user_id=00670dad22ad46efbb266d8c1c7e427b)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:13.630404 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa0ba61-6f8c-4938-acdb-887a2cb49162 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:13.729820 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-1fa0ba61-6f8c-4938-acdb-887a2cb49162 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00670dad22ad46efbb266d8c1c7e427b'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:01:13.746223 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-1fa0ba61-6f8c-4938-acdb-887a2cb49162 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00670dad22ad46efbb266d8c1c7e427b'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:01:13.757419 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-1fa0ba61-6f8c-4938-acdb-887a2cb49162 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00670dad22ad46efbb266d8c1c7e427b'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:01:13.760183 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-1fa0ba61-6f8c-4938-acdb-887a2cb49162 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00670dad22ad46efbb266d8c1c7e427b'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:01:13.760183 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-1fa0ba61-6f8c-4938-acdb-887a2cb49162 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00670dad22ad46efbb266d8c1c7e427b'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:01:13.830538 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1017/2028] 10.222.0.64 () {58 vars in 1224 bytes} [Tue Aug 2 10:01:13 2022] DELETE /identity/v3/users/00670dad22ad46efbb266d8c1c7e427b => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:01:14.935022 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b996399-55d6-4d3e-8cb1-90a8bbd831a9 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:14.936623 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-3b996399-55d6-4d3e-8cb1-90a8bbd831a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:14.960982 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b996399-55d6-4d3e-8cb1-90a8bbd831a9 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:15.069214 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:15.089613 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fa7d9e-b985-4ee0-8c6f-6077c689d300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:15.091564 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fa7d9e-b985-4ee0-8c6f-6077c689d300 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:15.092084 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fa7d9e-b985-4ee0-8c6f-6077c689d300 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:15.092615 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fa7d9e-b985-4ee0-8c6f-6077c689d300 admin admin] PATH_INFO: `/v3/projects/5f234bc78f854284a5a21f68297c16a2` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:15.095356 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fa7d9e-b985-4ee0-8c6f-6077c689d300 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f234bc78f854284a5a21f68297c16a2)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:15.098274 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fa7d9e-b985-4ee0-8c6f-6077c689d300 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:15.181342 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-60fa7d9e-b985-4ee0-8c6f-6077c689d300 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f234bc78f854284a5a21f68297c16a2', 'request_id': 'req-60fa7d9e-b985-4ee0-8c6f-6077c689d300'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:01:15.187900 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-60fa7d9e-b985-4ee0-8c6f-6077c689d300 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f234bc78f854284a5a21f68297c16a2', 'request_id': 'req-60fa7d9e-b985-4ee0-8c6f-6077c689d300'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:01:15.197479 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-60fa7d9e-b985-4ee0-8c6f-6077c689d300 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f234bc78f854284a5a21f68297c16a2', 'request_id': 'req-60fa7d9e-b985-4ee0-8c6f-6077c689d300'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:01:15.219145 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-60fa7d9e-b985-4ee0-8c6f-6077c689d300 admin admin] The token cache is being invalidate because project 5f234bc78f854284a5a21f68297c16a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80154) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:01:15.219583 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-60fa7d9e-b985-4ee0-8c6f-6077c689d300 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:01:15.221218 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-60fa7d9e-b985-4ee0-8c6f-6077c689d300 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:01:15.224353 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1012/2029] 10.222.0.64 () {58 vars in 1233 bytes} [Tue Aug 2 10:01:14 2022] DELETE /identity/v3/projects/5f234bc78f854284a5a21f68297c16a2 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:01:15.230727 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4ea023-3b00-4b2e-983d-d92e85c97d59 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:15.231277 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4ea023-3b00-4b2e-983d-d92e85c97d59 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:15.231778 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4ea023-3b00-4b2e-983d-d92e85c97d59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:15.259956 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-3f4ea023-3b00-4b2e-983d-d92e85c97d59 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:15.262017 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-3f4ea023-3b00-4b2e-983d-d92e85c97d59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:15.350644 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1018/2030] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:01:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:01:15.359496 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd446f64-c017-49b2-8c0e-f03c5279a213 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:15.360044 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd446f64-c017-49b2-8c0e-f03c5279a213 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:15.360622 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd446f64-c017-49b2-8c0e-f03c5279a213 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:15.393192 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-dd446f64-c017-49b2-8c0e-f03c5279a213 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:15.395698 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-dd446f64-c017-49b2-8c0e-f03c5279a213 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:15.408298 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1013/2031] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:01:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:01:15.412235 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4ea023-3b00-4b2e-983d-d92e85c97d59 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:15.426202 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4ea023-3b00-4b2e-983d-d92e85c97d59 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:15.443528 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9a6dc8-58f4-43cb-a561-409b6d7208ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:15.445939 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9a6dc8-58f4-43cb-a561-409b6d7208ed admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:15.446352 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9a6dc8-58f4-43cb-a561-409b6d7208ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:15.446738 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9a6dc8-58f4-43cb-a561-409b6d7208ed admin admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:15.447744 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9a6dc8-58f4-43cb-a561-409b6d7208ed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:15.448166 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9a6dc8-58f4-43cb-a561-409b6d7208ed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:15.450212 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9a6dc8-58f4-43cb-a561-409b6d7208ed admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:15.452612 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:15.477749 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1019/2032] 10.222.0.64 () {58 vars in 1153 bytes} [Tue Aug 2 10:01:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:15.477969 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd446f64-c017-49b2-8c0e-f03c5279a213 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:15.495270 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd446f64-c017-49b2-8c0e-f03c5279a213 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:15.501669 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a472ef-eefb-47eb-b0b3-407106d1a118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:15.502309 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a472ef-eefb-47eb-b0b3-407106d1a118 admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:15.502653 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a472ef-eefb-47eb-b0b3-407106d1a118 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:15.502929 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a472ef-eefb-47eb-b0b3-407106d1a118 admin admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:15.503625 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a472ef-eefb-47eb-b0b3-407106d1a118 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:15.505818 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a472ef-eefb-47eb-b0b3-407106d1a118 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:15.578734 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1014/2033] 10.222.0.64 () {60 vars in 1153 bytes} [Tue Aug 2 10:01:15 2022] POST /identity/v3/projects => generated 399 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:01:15.582788 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9a6dc8-58f4-43cb-a561-409b6d7208ed admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:15.599025 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9a6dc8-58f4-43cb-a561-409b6d7208ed admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:15.698083 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9163ad70-e60a-44f3-89bf-c6bd2d23a74d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:15.699527 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9163ad70-e60a-44f3-89bf-c6bd2d23a74d admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:15.699527 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9163ad70-e60a-44f3-89bf-c6bd2d23a74d admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:15.699527 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9163ad70-e60a-44f3-89bf-c6bd2d23a74d admin admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:15.700462 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9163ad70-e60a-44f3-89bf-c6bd2d23a74d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:15.702778 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9163ad70-e60a-44f3-89bf-c6bd2d23a74d admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:15.731324 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1020/2034] 10.222.0.64 () {60 vars in 1144 bytes} [Tue Aug 2 10:01:15 2022] POST /identity/v3/users => generated 300 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:01:15.736467 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a472ef-eefb-47eb-b0b3-407106d1a118 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:15.747611 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a472ef-eefb-47eb-b0b3-407106d1a118 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:15.759337 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3a6802-0497-4185-960b-717866db1246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:15.760146 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3a6802-0497-4185-960b-717866db1246 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:15.760587 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3a6802-0497-4185-960b-717866db1246 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:15.760930 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3a6802-0497-4185-960b-717866db1246 admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:15.761738 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3a6802-0497-4185-960b-717866db1246 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:15.763708 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3a6802-0497-4185-960b-717866db1246 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:15.775895 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1015/2035] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:01:15 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:01:15.782931 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9163ad70-e60a-44f3-89bf-c6bd2d23a74d admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:15.793302 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9163ad70-e60a-44f3-89bf-c6bd2d23a74d admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:15.798825 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440b0293-ba3e-4d1d-802f-48b0f2bf8973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:15.799446 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440b0293-ba3e-4d1d-802f-48b0f2bf8973 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:15.799734 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440b0293-ba3e-4d1d-802f-48b0f2bf8973 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:15.800031 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440b0293-ba3e-4d1d-802f-48b0f2bf8973 admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:15.800763 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440b0293-ba3e-4d1d-802f-48b0f2bf8973 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:15.802978 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440b0293-ba3e-4d1d-802f-48b0f2bf8973 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:15.816930 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1021/2036] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:01:15 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:01:15.826279 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3a6802-0497-4185-960b-717866db1246 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:15.844385 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3a6802-0497-4185-960b-717866db1246 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:15.849085 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f190a8-d7f0-41f4-b122-d49309acb57e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:15.849853 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f190a8-d7f0-41f4-b122-d49309acb57e admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:15.850145 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f190a8-d7f0-41f4-b122-d49309acb57e admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:15.850429 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f190a8-d7f0-41f4-b122-d49309acb57e admin admin] PATH_INFO: `/v3/projects/7a258b92c80b401f9d6fefc5ca158c44/users/4a0a4accf1734d459d524b06f636129f/roles/66c1074780954f8e87ed11a5c4e8c36a` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:15.866163 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f190a8-d7f0-41f4-b122-d49309acb57e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a258b92c80b401f9d6fefc5ca158c44, user_id=4a0a4accf1734d459d524b06f636129f, role_id=66c1074780954f8e87ed11a5c4e8c36a)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:15.868976 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f190a8-d7f0-41f4-b122-d49309acb57e admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:15.886961 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1016/2037] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 10:01:15 2022] PUT /identity/v3/projects/7a258b92c80b401f9d6fefc5ca158c44/users/4a0a4accf1734d459d524b06f636129f/roles/66c1074780954f8e87ed11a5c4e8c36a => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:01:15.900187 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b48555f-39dd-4417-bdb7-e392a4dbf1d1 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:15.901142 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b48555f-39dd-4417-bdb7-e392a4dbf1d1 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:15.901744 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b48555f-39dd-4417-bdb7-e392a4dbf1d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:15.934784 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-3b48555f-39dd-4417-bdb7-e392a4dbf1d1 None None] MFA Rules not processed for user `4a0a4accf1734d459d524b06f636129f`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:15.936178 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-3b48555f-39dd-4417-bdb7-e392a4dbf1d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:16.002055 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1022/2038] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:01:15 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:01:16.013629 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f190a8-d7f0-41f4-b122-d49309acb57e admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:16.020701 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-00f190a8-d7f0-41f4-b122-d49309acb57e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:16.108859 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f190a8-d7f0-41f4-b122-d49309acb57e admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:16.113528 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21bf459-3c82-48e3-af6d-122e486c9138 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d7de62b9ee4039bdc79627ac7fb499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:16.114367 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21bf459-3c82-48e3-af6d-122e486c9138 service cinder] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:16.114761 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21bf459-3c82-48e3-af6d-122e486c9138 service cinder] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:16.115418 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21bf459-3c82-48e3-af6d-122e486c9138 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:16.125944 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21bf459-3c82-48e3-af6d-122e486c9138 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:16.127487 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21bf459-3c82-48e3-af6d-122e486c9138 service cinder] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:16.144700 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1017/2039] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:01:16 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:01:25.998649 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=69,keystone:INSERT=3 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:26.135109 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=82,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:26.574159 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b48555f-39dd-4417-bdb7-e392a4dbf1d1 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:26.637470 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b48555f-39dd-4417-bdb7-e392a4dbf1d1 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:26.646494 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce8d257-389b-4dfb-9475-188b155d1c6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:26.647154 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce8d257-389b-4dfb-9475-188b155d1c6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:26.647435 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce8d257-389b-4dfb-9475-188b155d1c6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:26.647711 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce8d257-389b-4dfb-9475-188b155d1c6a admin admin] PATH_INFO: `/v3/users/4a0a4accf1734d459d524b06f636129f` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:26.648916 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce8d257-389b-4dfb-9475-188b155d1c6a admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a0a4accf1734d459d524b06f636129f)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:26.651226 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce8d257-389b-4dfb-9475-188b155d1c6a admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:26.714145 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-2ce8d257-389b-4dfb-9475-188b155d1c6a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a0a4accf1734d459d524b06f636129f'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:01:26.730705 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-2ce8d257-389b-4dfb-9475-188b155d1c6a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a0a4accf1734d459d524b06f636129f'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:01:26.739144 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-2ce8d257-389b-4dfb-9475-188b155d1c6a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a0a4accf1734d459d524b06f636129f'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:01:26.739768 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-2ce8d257-389b-4dfb-9475-188b155d1c6a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a0a4accf1734d459d524b06f636129f'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:01:26.740419 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-2ce8d257-389b-4dfb-9475-188b155d1c6a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a0a4accf1734d459d524b06f636129f'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:01:26.758524 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1023/2040] 10.222.0.64 () {58 vars in 1224 bytes} [Tue Aug 2 10:01:26 2022] DELETE /identity/v3/users/4a0a4accf1734d459d524b06f636129f => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:01:26.763170 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a52b53-377e-4d88-ae1d-bc3301c2170a None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:26.763463 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a52b53-377e-4d88-ae1d-bc3301c2170a None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:26.763767 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a52b53-377e-4d88-ae1d-bc3301c2170a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:26.782603 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-f6a52b53-377e-4d88-ae1d-bc3301c2170a None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:26.783865 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-f6a52b53-377e-4d88-ae1d-bc3301c2170a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:26.844588 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1018/2041] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:01:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:01:26.854440 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce8d257-389b-4dfb-9475-188b155d1c6a admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:26.856010 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-2ce8d257-389b-4dfb-9475-188b155d1c6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:26.918980 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce8d257-389b-4dfb-9475-188b155d1c6a admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:26.922638 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f6d384-235e-4a53-bba1-7c5fc60f97ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:26.923315 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f6d384-235e-4a53-bba1-7c5fc60f97ae service neutron] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:26.923600 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f6d384-235e-4a53-bba1-7c5fc60f97ae service neutron] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:26.924434 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f6d384-235e-4a53-bba1-7c5fc60f97ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:26.934477 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f6d384-235e-4a53-bba1-7c5fc60f97ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:26.935939 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f6d384-235e-4a53-bba1-7c5fc60f97ae service neutron] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:26.946091 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1024/2042] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:01:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:01:27.314006 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a52b53-377e-4d88-ae1d-bc3301c2170a None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:27.315515 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-f6a52b53-377e-4d88-ae1d-bc3301c2170a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:27.327840 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a52b53-377e-4d88-ae1d-bc3301c2170a None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:27.333249 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dafabc-22b4-4841-993c-16a810c7d3f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:27.333906 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dafabc-22b4-4841-993c-16a810c7d3f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:27.334198 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dafabc-22b4-4841-993c-16a810c7d3f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:27.334477 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dafabc-22b4-4841-993c-16a810c7d3f3 admin admin] PATH_INFO: `/v3/projects/7a258b92c80b401f9d6fefc5ca158c44` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:27.341067 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dafabc-22b4-4841-993c-16a810c7d3f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a258b92c80b401f9d6fefc5ca158c44)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:27.343100 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dafabc-22b4-4841-993c-16a810c7d3f3 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:27.392165 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-b4dafabc-22b4-4841-993c-16a810c7d3f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a258b92c80b401f9d6fefc5ca158c44', 'request_id': 'req-b4dafabc-22b4-4841-993c-16a810c7d3f3'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:01:27.398179 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-b4dafabc-22b4-4841-993c-16a810c7d3f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a258b92c80b401f9d6fefc5ca158c44', 'request_id': 'req-b4dafabc-22b4-4841-993c-16a810c7d3f3'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:01:27.404404 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-b4dafabc-22b4-4841-993c-16a810c7d3f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a258b92c80b401f9d6fefc5ca158c44', 'request_id': 'req-b4dafabc-22b4-4841-993c-16a810c7d3f3'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:01:27.418875 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-b4dafabc-22b4-4841-993c-16a810c7d3f3 admin admin] The token cache is being invalidate because project 7a258b92c80b401f9d6fefc5ca158c44 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80154) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:01:27.419188 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-b4dafabc-22b4-4841-993c-16a810c7d3f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:01:27.419796 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-b4dafabc-22b4-4841-993c-16a810c7d3f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:01:27.421343 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1019/2043] 10.222.0.64 () {58 vars in 1233 bytes} [Tue Aug 2 10:01:27 2022] DELETE /identity/v3/projects/7a258b92c80b401f9d6fefc5ca158c44 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:01:27.426341 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287d8ee6-423d-4b92-839f-86afbec28789 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:27.426794 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287d8ee6-423d-4b92-839f-86afbec28789 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:27.426959 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287d8ee6-423d-4b92-839f-86afbec28789 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:27.446833 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-287d8ee6-423d-4b92-839f-86afbec28789 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:27.447978 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-287d8ee6-423d-4b92-839f-86afbec28789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:27.473103 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:27.518884 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1025/2044] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:01:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:01:27.526880 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35da9495-d259-4157-ae3f-22077ffcdef0 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:27.527185 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35da9495-d259-4157-ae3f-22077ffcdef0 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:27.527451 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35da9495-d259-4157-ae3f-22077ffcdef0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:27.547323 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-35da9495-d259-4157-ae3f-22077ffcdef0 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:27.548559 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-35da9495-d259-4157-ae3f-22077ffcdef0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:27.556718 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1020/2045] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:01:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:01:27.560930 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287d8ee6-423d-4b92-839f-86afbec28789 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:27.570946 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287d8ee6-423d-4b92-839f-86afbec28789 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:27.576305 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426795a7-5e11-41f3-b0d5-cb7788d044e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:27.576923 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426795a7-5e11-41f3-b0d5-cb7788d044e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:27.577206 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426795a7-5e11-41f3-b0d5-cb7788d044e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:27.577507 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426795a7-5e11-41f3-b0d5-cb7788d044e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:27.578239 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426795a7-5e11-41f3-b0d5-cb7788d044e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:27.578544 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426795a7-5e11-41f3-b0d5-cb7788d044e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:27.580694 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426795a7-5e11-41f3-b0d5-cb7788d044e0 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:27.593683 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1026/2046] 10.222.0.64 () {58 vars in 1153 bytes} [Tue Aug 2 10:01:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:27.598822 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35da9495-d259-4157-ae3f-22077ffcdef0 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:27.608146 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35da9495-d259-4157-ae3f-22077ffcdef0 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:27.613082 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dce983a-2e79-4257-9496-847f99a406c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:27.613707 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dce983a-2e79-4257-9496-847f99a406c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:27.613996 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dce983a-2e79-4257-9496-847f99a406c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:27.614271 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dce983a-2e79-4257-9496-847f99a406c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:27.614979 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dce983a-2e79-4257-9496-847f99a406c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:27.616949 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dce983a-2e79-4257-9496-847f99a406c2 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:27.646196 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1021/2047] 10.222.0.64 () {60 vars in 1153 bytes} [Tue Aug 2 10:01:27 2022] POST /identity/v3/projects => generated 377 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:01:27.650604 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426795a7-5e11-41f3-b0d5-cb7788d044e0 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:27.661065 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426795a7-5e11-41f3-b0d5-cb7788d044e0 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:27.716188 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5091b5-141e-409e-8b68-6d31ff90018a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:27.716747 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5091b5-141e-409e-8b68-6d31ff90018a admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:27.717084 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5091b5-141e-409e-8b68-6d31ff90018a admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:27.717373 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5091b5-141e-409e-8b68-6d31ff90018a admin admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:27.718077 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5091b5-141e-409e-8b68-6d31ff90018a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:27.719988 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5091b5-141e-409e-8b68-6d31ff90018a admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:27.754715 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1027/2048] 10.222.0.64 () {60 vars in 1144 bytes} [Tue Aug 2 10:01:27 2022] POST /identity/v3/users => generated 289 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:01:27.755024 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dce983a-2e79-4257-9496-847f99a406c2 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:27.768394 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dce983a-2e79-4257-9496-847f99a406c2 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:27.775575 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc691a6-35d9-414c-a54a-f426d11a0f2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:27.776209 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc691a6-35d9-414c-a54a-f426d11a0f2d admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:27.776520 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc691a6-35d9-414c-a54a-f426d11a0f2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:27.776799 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc691a6-35d9-414c-a54a-f426d11a0f2d admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:27.777958 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc691a6-35d9-414c-a54a-f426d11a0f2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:27.780824 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc691a6-35d9-414c-a54a-f426d11a0f2d admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:27.797018 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1022/2049] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:01:27 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:01:27.797693 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5091b5-141e-409e-8b68-6d31ff90018a admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:27.808372 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5091b5-141e-409e-8b68-6d31ff90018a admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:27.813772 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f85b89d-cfca-4711-908f-7c8bdb420652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:27.815182 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f85b89d-cfca-4711-908f-7c8bdb420652 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:27.815673 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f85b89d-cfca-4711-908f-7c8bdb420652 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:27.816052 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f85b89d-cfca-4711-908f-7c8bdb420652 admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:27.817079 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f85b89d-cfca-4711-908f-7c8bdb420652 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:27.819512 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f85b89d-cfca-4711-908f-7c8bdb420652 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:27.831770 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1028/2050] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:01:27 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:01:27.836029 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc691a6-35d9-414c-a54a-f426d11a0f2d admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:27.846513 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc691a6-35d9-414c-a54a-f426d11a0f2d admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:27.851754 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081f79f2-dcf7-47da-8782-ba3284ce1d92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:27.852582 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081f79f2-dcf7-47da-8782-ba3284ce1d92 admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:27.852948 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081f79f2-dcf7-47da-8782-ba3284ce1d92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:27.853292 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081f79f2-dcf7-47da-8782-ba3284ce1d92 admin admin] PATH_INFO: `/v3/projects/0ef2c95a509c446e870129685685e1ac/users/7739a00c6524467d8fd139183c6667cf/roles/66c1074780954f8e87ed11a5c4e8c36a` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:27.870682 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081f79f2-dcf7-47da-8782-ba3284ce1d92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ef2c95a509c446e870129685685e1ac, user_id=7739a00c6524467d8fd139183c6667cf, role_id=66c1074780954f8e87ed11a5c4e8c36a)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:27.873601 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081f79f2-dcf7-47da-8782-ba3284ce1d92 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:27.889906 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1023/2051] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 10:01:27 2022] PUT /identity/v3/projects/0ef2c95a509c446e870129685685e1ac/users/7739a00c6524467d8fd139183c6667cf/roles/66c1074780954f8e87ed11a5c4e8c36a => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:01:27.896072 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378293cf-99dc-43d8-b60f-069147751e84 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:27.896448 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378293cf-99dc-43d8-b60f-069147751e84 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:27.896729 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378293cf-99dc-43d8-b60f-069147751e84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:27.916897 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-378293cf-99dc-43d8-b60f-069147751e84 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:27.918133 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-378293cf-99dc-43d8-b60f-069147751e84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:27.987045 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1029/2052] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:01:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:01:27.996487 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081f79f2-dcf7-47da-8782-ba3284ce1d92 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:27.998182 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-081f79f2-dcf7-47da-8782-ba3284ce1d92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:28.020184 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:28.057476 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081f79f2-dcf7-47da-8782-ba3284ce1d92 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:28.061385 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9e063e-3217-4ab0-ac73-44a6a0207cc3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:28.062001 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9e063e-3217-4ab0-ac73-44a6a0207cc3 service neutron] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:28.062282 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9e063e-3217-4ab0-ac73-44a6a0207cc3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:28.062551 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9e063e-3217-4ab0-ac73-44a6a0207cc3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:28.071752 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9e063e-3217-4ab0-ac73-44a6a0207cc3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:28.073202 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9e063e-3217-4ab0-ac73-44a6a0207cc3 service neutron] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:28.083306 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1024/2053] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:01:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:01:33.206148 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e84517-1886-4e42-9994-d728855c6b42 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:33.206860 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e84517-1886-4e42-9994-d728855c6b42 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:33.206860 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e84517-1886-4e42-9994-d728855c6b42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:33.250203 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-b7e84517-1886-4e42-9994-d728855c6b42 None None] MFA Rules not processed for user `7739a00c6524467d8fd139183c6667cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:33.252827 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-b7e84517-1886-4e42-9994-d728855c6b42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:33.278727 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:33.385104 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1030/2054] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:01:33 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:01:33.389762 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9e063e-3217-4ab0-ac73-44a6a0207cc3 service neutron] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:33.399457 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9e063e-3217-4ab0-ac73-44a6a0207cc3 service neutron] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:33.404182 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249c20f2-9544-43e7-a0b9-f98e57670e86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:33.404832 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249c20f2-9544-43e7-a0b9-f98e57670e86 admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:33.405115 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249c20f2-9544-43e7-a0b9-f98e57670e86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:33.405387 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249c20f2-9544-43e7-a0b9-f98e57670e86 admin admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:33.406065 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249c20f2-9544-43e7-a0b9-f98e57670e86 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:33.408603 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249c20f2-9544-43e7-a0b9-f98e57670e86 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:33.441579 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1025/2055] 10.222.0.64 () {60 vars in 1153 bytes} [Tue Aug 2 10:01:33 2022] POST /identity/v3/projects => generated 379 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:01:33.443177 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e84517-1886-4e42-9994-d728855c6b42 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:33.455961 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e84517-1886-4e42-9994-d728855c6b42 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:33.521547 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74875a22-4a5b-40ed-b137-840c4018ae7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:33.522103 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74875a22-4a5b-40ed-b137-840c4018ae7d admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:33.522537 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74875a22-4a5b-40ed-b137-840c4018ae7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:33.522980 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74875a22-4a5b-40ed-b137-840c4018ae7d admin admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:33.523791 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74875a22-4a5b-40ed-b137-840c4018ae7d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:33.526354 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74875a22-4a5b-40ed-b137-840c4018ae7d admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:33.552942 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1031/2056] 10.222.0.64 () {60 vars in 1144 bytes} [Tue Aug 2 10:01:33 2022] POST /identity/v3/users => generated 296 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:01:33.557006 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249c20f2-9544-43e7-a0b9-f98e57670e86 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:33.568860 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249c20f2-9544-43e7-a0b9-f98e57670e86 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:33.573658 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1b921d-a0c9-438f-ae62-27fba2618d3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:33.574287 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1b921d-a0c9-438f-ae62-27fba2618d3b admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:33.574597 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1b921d-a0c9-438f-ae62-27fba2618d3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:33.574851 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1b921d-a0c9-438f-ae62-27fba2618d3b admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:33.575811 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1b921d-a0c9-438f-ae62-27fba2618d3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:33.577453 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1b921d-a0c9-438f-ae62-27fba2618d3b admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:33.589044 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1026/2057] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:01:33 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:01:33.594184 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74875a22-4a5b-40ed-b137-840c4018ae7d admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:33.609079 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74875a22-4a5b-40ed-b137-840c4018ae7d admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:33.614701 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946b11e2-881c-486a-8e8f-54160e16f7df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:33.615379 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946b11e2-881c-486a-8e8f-54160e16f7df admin admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:33.615691 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946b11e2-881c-486a-8e8f-54160e16f7df admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:33.615974 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946b11e2-881c-486a-8e8f-54160e16f7df admin admin] PATH_INFO: `/v3/projects/c9234311754747439023d17adb301608/users/dc2ef0f65b6643eca00c2106f8f8c5d5/roles/1000cf318fa14915b8091599eaead5de` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:33.641258 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946b11e2-881c-486a-8e8f-54160e16f7df admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9234311754747439023d17adb301608, user_id=dc2ef0f65b6643eca00c2106f8f8c5d5, role_id=1000cf318fa14915b8091599eaead5de)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:33.646846 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946b11e2-881c-486a-8e8f-54160e16f7df admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:33.663117 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1032/2058] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 10:01:33 2022] PUT /identity/v3/projects/c9234311754747439023d17adb301608/users/dc2ef0f65b6643eca00c2106f8f8c5d5/roles/1000cf318fa14915b8091599eaead5de => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:01:33.668353 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1b921d-a0c9-438f-ae62-27fba2618d3b admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:33.680949 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1b921d-a0c9-438f-ae62-27fba2618d3b admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:33.779176 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a078f9-420e-4860-8a1c-5c6dfb75b523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:33.779830 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a078f9-420e-4860-8a1c-5c6dfb75b523 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:33.780113 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a078f9-420e-4860-8a1c-5c6dfb75b523 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:33.780436 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a078f9-420e-4860-8a1c-5c6dfb75b523 admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:33.781207 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a078f9-420e-4860-8a1c-5c6dfb75b523 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:33.783429 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a078f9-420e-4860-8a1c-5c6dfb75b523 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:33.796039 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1027/2059] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:01:33 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:01:33.800711 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946b11e2-881c-486a-8e8f-54160e16f7df admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:33.813125 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946b11e2-881c-486a-8e8f-54160e16f7df admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:33.818622 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bc4640-5eae-47c3-b99e-42c2e92c770f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:33.819320 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bc4640-5eae-47c3-b99e-42c2e92c770f admin admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:33.819757 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bc4640-5eae-47c3-b99e-42c2e92c770f admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:33.820041 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bc4640-5eae-47c3-b99e-42c2e92c770f admin admin] PATH_INFO: `/v3/projects/c9234311754747439023d17adb301608/users/dc2ef0f65b6643eca00c2106f8f8c5d5/roles/1000cf318fa14915b8091599eaead5de` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:33.822310 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bc4640-5eae-47c3-b99e-42c2e92c770f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9234311754747439023d17adb301608, user_id=dc2ef0f65b6643eca00c2106f8f8c5d5, role_id=1000cf318fa14915b8091599eaead5de)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:33.830030 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bc4640-5eae-47c3-b99e-42c2e92c770f admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:33.854447 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1033/2060] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 10:01:33 2022] PUT /identity/v3/projects/c9234311754747439023d17adb301608/users/dc2ef0f65b6643eca00c2106f8f8c5d5/roles/1000cf318fa14915b8091599eaead5de => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:01:40.137237 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f71905-6adb-4579-87ea-5093177e9081 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:40.138114 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f71905-6adb-4579-87ea-5093177e9081 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:40.138885 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f71905-6adb-4579-87ea-5093177e9081 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:40.181878 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-83f71905-6adb-4579-87ea-5093177e9081 None None] MFA Rules not processed for user `dc2ef0f65b6643eca00c2106f8f8c5d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:40.183093 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-83f71905-6adb-4579-87ea-5093177e9081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:40.260441 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1028/2061] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:01:40 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:01:40.279164 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bc4640-5eae-47c3-b99e-42c2e92c770f admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:40.281667 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-36bc4640-5eae-47c3-b99e-42c2e92c770f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:40.369861 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bc4640-5eae-47c3-b99e-42c2e92c770f admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:40.375468 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ad6c04-9274-4ab3-86f8-7b33de1f192e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99655e1f9ca24472a86cd26b0dbd1fc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:40.376095 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ad6c04-9274-4ab3-86f8-7b33de1f192e service nova] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:40.376931 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ad6c04-9274-4ab3-86f8-7b33de1f192e service nova] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:40.377209 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ad6c04-9274-4ab3-86f8-7b33de1f192e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:40.382711 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:40.441219 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ad6c04-9274-4ab3-86f8-7b33de1f192e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:40.442691 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ad6c04-9274-4ab3-86f8-7b33de1f192e service nova] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:40.453698 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1034/2062] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:01:40 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:01:48.082967 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd02d92-bd45-4c05-8f84-29d7815e9f35 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:48.083460 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd02d92-bd45-4c05-8f84-29d7815e9f35 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:48.083562 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd02d92-bd45-4c05-8f84-29d7815e9f35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:48.107336 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-1dd02d92-bd45-4c05-8f84-29d7815e9f35 None None] MFA Rules not processed for user `5338bc8c594e4bc9bc41019226bf8d87`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:48.108618 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-1dd02d92-bd45-4c05-8f84-29d7815e9f35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:48.115004 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:48.174118 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1029/2063] 10.222.0.64 () {58 vars in 1009 bytes} [Tue Aug 2 10:01:48 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:01:48.190708 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225c974c-adcc-431d-ad44-705b6c2a8621 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:48.191128 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225c974c-adcc-431d-ad44-705b6c2a8621 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:48.191382 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225c974c-adcc-431d-ad44-705b6c2a8621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:48.212183 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-225c974c-adcc-431d-ad44-705b6c2a8621 None None] MFA Rules not processed for user `ede2486b45f44ac3b0bdab2d40695bee`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:48.213628 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-225c974c-adcc-431d-ad44-705b6c2a8621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:48.224861 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd02d92-bd45-4c05-8f84-29d7815e9f35 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:48.226680 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-1dd02d92-bd45-4c05-8f84-29d7815e9f35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:48.282715 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1035/2064] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:01:48 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:01:48.306287 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd02d92-bd45-4c05-8f84-29d7815e9f35 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:48.311583 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab735ad5-6c87-412a-b7be-222a0db7ce00 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a371e3df3764a4cbb1b8368aa74bd65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:48.312200 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab735ad5-6c87-412a-b7be-222a0db7ce00 service swift] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:48.312514 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab735ad5-6c87-412a-b7be-222a0db7ce00 service swift] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:48.312787 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab735ad5-6c87-412a-b7be-222a0db7ce00 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:48.321208 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab735ad5-6c87-412a-b7be-222a0db7ce00 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:48.322775 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab735ad5-6c87-412a-b7be-222a0db7ce00 service swift] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:48.330940 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1030/2065] 10.222.0.64 () {60 vars in 1384 bytes} [Tue Aug 2 10:01:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 02 10:01:48.999552 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0469643-17ef-4552-8100-fdab12702779 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:48.999851 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0469643-17ef-4552-8100-fdab12702779 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:49.000572 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0469643-17ef-4552-8100-fdab12702779 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:49.042740 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-b0469643-17ef-4552-8100-fdab12702779 None None] MFA Rules not processed for user `5338bc8c594e4bc9bc41019226bf8d87`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:49.044863 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-b0469643-17ef-4552-8100-fdab12702779 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:49.071530 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1036/2066] 10.222.0.64 () {58 vars in 967 bytes} [Tue Aug 2 10:01:48 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:01:49.072066 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab735ad5-6c87-412a-b7be-222a0db7ce00 service swift] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:49.094692 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab735ad5-6c87-412a-b7be-222a0db7ce00 service swift] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:49.106801 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6cbd71-b410-4c19-ac08-63d375530d28 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a371e3df3764a4cbb1b8368aa74bd65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:49.107889 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6cbd71-b410-4c19-ac08-63d375530d28 service swift] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:49.113455 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6cbd71-b410-4c19-ac08-63d375530d28 service swift] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:49.113964 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6cbd71-b410-4c19-ac08-63d375530d28 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:49.153281 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6cbd71-b410-4c19-ac08-63d375530d28 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:49.154882 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6cbd71-b410-4c19-ac08-63d375530d28 service swift] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:49.177671 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1031/2067] 10.222.0.64 () {60 vars in 1384 bytes} [Tue Aug 2 10:01:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 02 10:01:49.495426 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f137640-2617-4245-b704-1bdb09050da3 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:49.495991 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f137640-2617-4245-b704-1bdb09050da3 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:49.496998 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f137640-2617-4245-b704-1bdb09050da3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:49.520177 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-8f137640-2617-4245-b704-1bdb09050da3 None None] MFA Rules not processed for user `5338bc8c594e4bc9bc41019226bf8d87`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:49.521599 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-8f137640-2617-4245-b704-1bdb09050da3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:49.536169 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1037/2068] 10.222.0.64 () {58 vars in 1009 bytes} [Tue Aug 2 10:01:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:01:49.549135 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6cbd71-b410-4c19-ac08-63d375530d28 service swift] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:49.561165 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6cbd71-b410-4c19-ac08-63d375530d28 service swift] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:49.566856 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a922e7f1-8deb-47b6-ac97-2f8803a038ae service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a371e3df3764a4cbb1b8368aa74bd65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:49.567803 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a922e7f1-8deb-47b6-ac97-2f8803a038ae service swift] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:49.568091 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a922e7f1-8deb-47b6-ac97-2f8803a038ae service swift] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:49.568927 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a922e7f1-8deb-47b6-ac97-2f8803a038ae service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:49.578152 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a922e7f1-8deb-47b6-ac97-2f8803a038ae service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:49.581003 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a922e7f1-8deb-47b6-ac97-2f8803a038ae service swift] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:49.597248 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1032/2069] 10.222.0.64 () {60 vars in 1384 bytes} [Tue Aug 2 10:01:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 02 10:01:49.979625 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb99a97-cb19-42a1-baf6-2e2179ff704a None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:49.979919 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb99a97-cb19-42a1-baf6-2e2179ff704a None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:49.980233 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb99a97-cb19-42a1-baf6-2e2179ff704a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:50.000511 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-0eb99a97-cb19-42a1-baf6-2e2179ff704a None None] MFA Rules not processed for user `5338bc8c594e4bc9bc41019226bf8d87`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:50.001791 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-0eb99a97-cb19-42a1-baf6-2e2179ff704a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:50.009028 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1038/2070] 10.222.0.64 () {58 vars in 967 bytes} [Tue Aug 2 10:01:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:01:50.021838 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a922e7f1-8deb-47b6-ac97-2f8803a038ae service swift] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:50.033992 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a922e7f1-8deb-47b6-ac97-2f8803a038ae service swift] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:50.039296 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e6a4ac-eb21-4fce-be97-04d8c3af4d3c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a371e3df3764a4cbb1b8368aa74bd65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:50.039907 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e6a4ac-eb21-4fce-be97-04d8c3af4d3c service swift] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:50.040189 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e6a4ac-eb21-4fce-be97-04d8c3af4d3c service swift] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:50.040498 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e6a4ac-eb21-4fce-be97-04d8c3af4d3c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:50.049624 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e6a4ac-eb21-4fce-be97-04d8c3af4d3c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:50.051225 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e6a4ac-eb21-4fce-be97-04d8c3af4d3c service swift] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:50.059555 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1033/2071] 10.222.0.64 () {60 vars in 1384 bytes} [Tue Aug 2 10:01:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 02 10:01:52.503286 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb99a97-cb19-42a1-baf6-2e2179ff704a None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:52.504887 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-0eb99a97-cb19-42a1-baf6-2e2179ff704a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:52.611014 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb99a97-cb19-42a1-baf6-2e2179ff704a None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:52.615685 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b344d7-d452-4704-940b-41defdea13f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:52.616384 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b344d7-d452-4704-940b-41defdea13f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:52.617942 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b344d7-d452-4704-940b-41defdea13f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:52.618609 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b344d7-d452-4704-940b-41defdea13f9 admin admin] PATH_INFO: `/v3/users/9ab0dff213d34f57b1c876a3e707573f` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:52.620336 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b344d7-d452-4704-940b-41defdea13f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ab0dff213d34f57b1c876a3e707573f)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:52.622548 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b344d7-d452-4704-940b-41defdea13f9 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:52.707404 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:52.724313 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-27b344d7-d452-4704-940b-41defdea13f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ab0dff213d34f57b1c876a3e707573f'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:01:52.743477 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-27b344d7-d452-4704-940b-41defdea13f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ab0dff213d34f57b1c876a3e707573f'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:01:52.755039 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-27b344d7-d452-4704-940b-41defdea13f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ab0dff213d34f57b1c876a3e707573f'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:01:52.755896 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-27b344d7-d452-4704-940b-41defdea13f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ab0dff213d34f57b1c876a3e707573f'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:01:52.757401 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-27b344d7-d452-4704-940b-41defdea13f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ab0dff213d34f57b1c876a3e707573f'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:01:52.795356 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1039/2072] 10.222.0.64 () {58 vars in 1224 bytes} [Tue Aug 2 10:01:52 2022] DELETE /identity/v3/users/9ab0dff213d34f57b1c876a3e707573f => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:01:52.796775 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4c9903-92d0-4ac9-91b8-38ab3168f657 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:52.797271 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4c9903-92d0-4ac9-91b8-38ab3168f657 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:52.797357 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4c9903-92d0-4ac9-91b8-38ab3168f657 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:52.834909 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-5d4c9903-92d0-4ac9-91b8-38ab3168f657 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:52.839067 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-5d4c9903-92d0-4ac9-91b8-38ab3168f657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:52.949914 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1034/2073] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:01:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:01:52.962673 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b344d7-d452-4704-940b-41defdea13f9 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:52.964434 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-27b344d7-d452-4704-940b-41defdea13f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:53.048687 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b344d7-d452-4704-940b-41defdea13f9 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:53.053198 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c37b18-4e09-45e1-be63-d7cf363f333f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:53.054409 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c37b18-4e09-45e1-be63-d7cf363f333f service neutron] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:53.054714 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c37b18-4e09-45e1-be63-d7cf363f333f service neutron] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:53.055000 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c37b18-4e09-45e1-be63-d7cf363f333f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:53.068064 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c37b18-4e09-45e1-be63-d7cf363f333f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:53.069701 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c37b18-4e09-45e1-be63-d7cf363f333f service neutron] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:53.081080 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1040/2074] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:01:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:01:53.519398 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4c9903-92d0-4ac9-91b8-38ab3168f657 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:53.520891 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-5d4c9903-92d0-4ac9-91b8-38ab3168f657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:53.541242 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4c9903-92d0-4ac9-91b8-38ab3168f657 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:53.560505 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1238301-7b81-4c7b-a65c-6e0c1b64b89e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:53.561696 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1238301-7b81-4c7b-a65c-6e0c1b64b89e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:53.562665 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1238301-7b81-4c7b-a65c-6e0c1b64b89e admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:53.564221 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1238301-7b81-4c7b-a65c-6e0c1b64b89e admin admin] PATH_INFO: `/v3/projects/fd2e958fa0914b3f89a55d009f7a5f2b` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:53.566693 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1238301-7b81-4c7b-a65c-6e0c1b64b89e admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd2e958fa0914b3f89a55d009f7a5f2b)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:53.568763 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1238301-7b81-4c7b-a65c-6e0c1b64b89e admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:53.570499 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:53.635763 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-a1238301-7b81-4c7b-a65c-6e0c1b64b89e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd2e958fa0914b3f89a55d009f7a5f2b', 'request_id': 'req-a1238301-7b81-4c7b-a65c-6e0c1b64b89e'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:01:53.645482 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-a1238301-7b81-4c7b-a65c-6e0c1b64b89e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd2e958fa0914b3f89a55d009f7a5f2b', 'request_id': 'req-a1238301-7b81-4c7b-a65c-6e0c1b64b89e'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:01:53.652636 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-a1238301-7b81-4c7b-a65c-6e0c1b64b89e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd2e958fa0914b3f89a55d009f7a5f2b', 'request_id': 'req-a1238301-7b81-4c7b-a65c-6e0c1b64b89e'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:01:53.666928 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-a1238301-7b81-4c7b-a65c-6e0c1b64b89e admin admin] The token cache is being invalidate because project fd2e958fa0914b3f89a55d009f7a5f2b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80154) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:01:53.667283 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-a1238301-7b81-4c7b-a65c-6e0c1b64b89e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:01:53.667894 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-a1238301-7b81-4c7b-a65c-6e0c1b64b89e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:01:53.681121 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65014864-4d04-4b0d-af15-7a9ad50f7f8f None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:53.681639 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65014864-4d04-4b0d-af15-7a9ad50f7f8f None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:53.681733 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65014864-4d04-4b0d-af15-7a9ad50f7f8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:53.687743 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1035/2075] 10.222.0.64 () {58 vars in 1233 bytes} [Tue Aug 2 10:01:53 2022] DELETE /identity/v3/projects/fd2e958fa0914b3f89a55d009f7a5f2b => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:01:53.710177 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-65014864-4d04-4b0d-af15-7a9ad50f7f8f None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:53.712111 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-65014864-4d04-4b0d-af15-7a9ad50f7f8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:53.834420 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1041/2076] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:01:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:01:53.847640 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20701ae0-aa3f-45d9-a6f4-8f0037c8f395 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:53.848516 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20701ae0-aa3f-45d9-a6f4-8f0037c8f395 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:53.848961 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20701ae0-aa3f-45d9-a6f4-8f0037c8f395 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:53.879367 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-20701ae0-aa3f-45d9-a6f4-8f0037c8f395 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:53.880990 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-20701ae0-aa3f-45d9-a6f4-8f0037c8f395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:53.903180 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1036/2077] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:01:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:01:53.907471 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65014864-4d04-4b0d-af15-7a9ad50f7f8f None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:53.917653 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65014864-4d04-4b0d-af15-7a9ad50f7f8f None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:53.923618 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04992dfd-b447-4cd2-bc9d-4a2d52807498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:53.924272 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04992dfd-b447-4cd2-bc9d-4a2d52807498 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:53.924632 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04992dfd-b447-4cd2-bc9d-4a2d52807498 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:53.924911 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04992dfd-b447-4cd2-bc9d-4a2d52807498 admin admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:53.925623 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04992dfd-b447-4cd2-bc9d-4a2d52807498 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:53.925930 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04992dfd-b447-4cd2-bc9d-4a2d52807498 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:53.927567 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04992dfd-b447-4cd2-bc9d-4a2d52807498 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:53.945443 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1042/2078] 10.222.0.64 () {58 vars in 1153 bytes} [Tue Aug 2 10:01:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:53.949834 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20701ae0-aa3f-45d9-a6f4-8f0037c8f395 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:53.971807 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20701ae0-aa3f-45d9-a6f4-8f0037c8f395 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:53.982800 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a611dbb-f5c6-4f0f-8fca-2e77c28f4799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:53.983821 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a611dbb-f5c6-4f0f-8fca-2e77c28f4799 admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:53.984177 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a611dbb-f5c6-4f0f-8fca-2e77c28f4799 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:53.984562 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a611dbb-f5c6-4f0f-8fca-2e77c28f4799 admin admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:53.985849 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a611dbb-f5c6-4f0f-8fca-2e77c28f4799 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:53.987927 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a611dbb-f5c6-4f0f-8fca-2e77c28f4799 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:54.017946 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1037/2079] 10.222.0.64 () {60 vars in 1153 bytes} [Tue Aug 2 10:01:53 2022] POST /identity/v3/projects => generated 391 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:01:54.022413 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04992dfd-b447-4cd2-bc9d-4a2d52807498 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:54.033297 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04992dfd-b447-4cd2-bc9d-4a2d52807498 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:54.077726 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:54.095411 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff2ee4a-9100-427d-8263-04ae1a6db04a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:54.096070 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff2ee4a-9100-427d-8263-04ae1a6db04a admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:54.096395 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff2ee4a-9100-427d-8263-04ae1a6db04a admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:54.096678 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff2ee4a-9100-427d-8263-04ae1a6db04a admin admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:54.097363 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff2ee4a-9100-427d-8263-04ae1a6db04a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:54.099656 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff2ee4a-9100-427d-8263-04ae1a6db04a admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:54.123943 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1043/2080] 10.222.0.64 () {60 vars in 1144 bytes} [Tue Aug 2 10:01:54 2022] POST /identity/v3/users => generated 296 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:01:54.128898 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a611dbb-f5c6-4f0f-8fca-2e77c28f4799 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:54.139243 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a611dbb-f5c6-4f0f-8fca-2e77c28f4799 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:54.143828 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5979363a-8e27-40c7-ac17-56d0bb61f573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:54.144456 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5979363a-8e27-40c7-ac17-56d0bb61f573 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:54.144783 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5979363a-8e27-40c7-ac17-56d0bb61f573 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:54.145054 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5979363a-8e27-40c7-ac17-56d0bb61f573 admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:54.146103 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5979363a-8e27-40c7-ac17-56d0bb61f573 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:54.149039 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5979363a-8e27-40c7-ac17-56d0bb61f573 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:54.162948 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1038/2081] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:01:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:01:54.164452 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff2ee4a-9100-427d-8263-04ae1a6db04a admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:54.175307 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff2ee4a-9100-427d-8263-04ae1a6db04a admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:54.179949 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308638e7-56af-472b-8e56-b1a9c937885b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:54.180592 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308638e7-56af-472b-8e56-b1a9c937885b admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:54.180878 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308638e7-56af-472b-8e56-b1a9c937885b admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:54.181153 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308638e7-56af-472b-8e56-b1a9c937885b admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:54.181853 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308638e7-56af-472b-8e56-b1a9c937885b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:54.183782 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308638e7-56af-472b-8e56-b1a9c937885b admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:54.198931 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1044/2082] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:01:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:01:54.213788 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5979363a-8e27-40c7-ac17-56d0bb61f573 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:54.224979 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5979363a-8e27-40c7-ac17-56d0bb61f573 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:54.230331 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c255aff-e041-42f1-a09e-75e6e112da00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:54.230998 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c255aff-e041-42f1-a09e-75e6e112da00 admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:54.232389 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c255aff-e041-42f1-a09e-75e6e112da00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:54.232389 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c255aff-e041-42f1-a09e-75e6e112da00 admin admin] PATH_INFO: `/v3/projects/3328d5ef35ba4d65bc089aeb3ecd16d8/users/79dd373c5e5546a59357479b9daf304f/roles/66c1074780954f8e87ed11a5c4e8c36a` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:54.250220 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c255aff-e041-42f1-a09e-75e6e112da00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3328d5ef35ba4d65bc089aeb3ecd16d8, user_id=79dd373c5e5546a59357479b9daf304f, role_id=66c1074780954f8e87ed11a5c4e8c36a)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:54.254040 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c255aff-e041-42f1-a09e-75e6e112da00 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:54.277007 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1039/2083] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 10:01:54 2022] PUT /identity/v3/projects/3328d5ef35ba4d65bc089aeb3ecd16d8/users/79dd373c5e5546a59357479b9daf304f/roles/66c1074780954f8e87ed11a5c4e8c36a => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:01:54.279583 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5e02a8-408a-4b40-b5de-f7d7e4a53992 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:54.279878 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5e02a8-408a-4b40-b5de-f7d7e4a53992 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:54.280204 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5e02a8-408a-4b40-b5de-f7d7e4a53992 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:54.302672 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010506eb-f301-43fc-ba7d-0fb07eaf1607 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:54.302958 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010506eb-f301-43fc-ba7d-0fb07eaf1607 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:54.303232 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010506eb-f301-43fc-ba7d-0fb07eaf1607 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:54.304827 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1040/2084] 10.222.0.64 () {54 vars in 921 bytes} [Tue Aug 2 10:01:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 10:01:54.311279 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-af5e02a8-408a-4b40-b5de-f7d7e4a53992 None None] MFA Rules not processed for user `79dd373c5e5546a59357479b9daf304f`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:54.312531 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-af5e02a8-408a-4b40-b5de-f7d7e4a53992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:54.336385 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79531c0c-399b-4ee3-b541-70c032875947 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:54.336872 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79531c0c-399b-4ee3-b541-70c032875947 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:54.337859 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79531c0c-399b-4ee3-b541-70c032875947 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:54.372898 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-79531c0c-399b-4ee3-b541-70c032875947 None None] MFA Rules not processed for user `b770f073c86b4d88837a8b1f796a8b1d`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:54.374573 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-79531c0c-399b-4ee3-b541-70c032875947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:54.401550 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1045/2085] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:01:54 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:01:54.416568 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5e02a8-408a-4b40-b5de-f7d7e4a53992 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:54.418534 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-af5e02a8-408a-4b40-b5de-f7d7e4a53992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:54.480145 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1041/2086] 10.222.0.64 () {58 vars in 1018 bytes} [Tue Aug 2 10:01:54 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:01:54.481261 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79531c0c-399b-4ee3-b541-70c032875947 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:54.486248 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-79531c0c-399b-4ee3-b541-70c032875947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:54.502657 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5e02a8-408a-4b40-b5de-f7d7e4a53992 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:54.507850 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c09ad26-c140-456b-81b0-f78bc52dc24f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d7de62b9ee4039bdc79627ac7fb499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:54.508568 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c09ad26-c140-456b-81b0-f78bc52dc24f service cinder] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:54.508757 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c09ad26-c140-456b-81b0-f78bc52dc24f service cinder] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:54.509034 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c09ad26-c140-456b-81b0-f78bc52dc24f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:54.518233 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c09ad26-c140-456b-81b0-f78bc52dc24f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:54.519669 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c09ad26-c140-456b-81b0-f78bc52dc24f service cinder] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:54.528926 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:54.529077 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1046/2087] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:01:54 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:01:54.530081 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c09ad26-c140-456b-81b0-f78bc52dc24f service cinder] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:54.531585 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-8c09ad26-c140-456b-81b0-f78bc52dc24f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:54.549700 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c09ad26-c140-456b-81b0-f78bc52dc24f service cinder] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:54.555235 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d7e1d2-73cb-4fb0-914b-c20ab1877f85 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b770f073c86b4d88837a8b1f796a8b1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:54.555841 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d7e1d2-73cb-4fb0-914b-c20ab1877f85 service placement] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:54.556324 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d7e1d2-73cb-4fb0-914b-c20ab1877f85 service placement] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:54.556575 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d7e1d2-73cb-4fb0-914b-c20ab1877f85 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:54.566370 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d7e1d2-73cb-4fb0-914b-c20ab1877f85 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:54.568094 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d7e1d2-73cb-4fb0-914b-c20ab1877f85 service placement] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:54.588399 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1047/2088] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:01:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:01:54.614443 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79531c0c-399b-4ee3-b541-70c032875947 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:54.619843 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7119a412-0ed3-418c-8de2-5bf4181b6d8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:54.620531 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7119a412-0ed3-418c-8de2-5bf4181b6d8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:54.620823 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7119a412-0ed3-418c-8de2-5bf4181b6d8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:54.621130 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7119a412-0ed3-418c-8de2-5bf4181b6d8f admin admin] PATH_INFO: `/v3/users/b49f064508cb481e820933549826b257` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:54.622901 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7119a412-0ed3-418c-8de2-5bf4181b6d8f admin admin] RBAC: Authorizing `identity:delete_user(user_id=b49f064508cb481e820933549826b257)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:54.625128 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7119a412-0ed3-418c-8de2-5bf4181b6d8f admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:54.682092 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-7119a412-0ed3-418c-8de2-5bf4181b6d8f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b49f064508cb481e820933549826b257'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:01:54.689812 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-7119a412-0ed3-418c-8de2-5bf4181b6d8f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b49f064508cb481e820933549826b257'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:01:54.690538 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-7119a412-0ed3-418c-8de2-5bf4181b6d8f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b49f064508cb481e820933549826b257'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:01:54.699112 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-7119a412-0ed3-418c-8de2-5bf4181b6d8f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b49f064508cb481e820933549826b257'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:01:54.712775 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-7119a412-0ed3-418c-8de2-5bf4181b6d8f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b49f064508cb481e820933549826b257'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:01:54.719809 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1042/2089] 10.222.0.64 () {58 vars in 1224 bytes} [Tue Aug 2 10:01:54 2022] DELETE /identity/v3/users/b49f064508cb481e820933549826b257 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:01:54.941484 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d7e1d2-73cb-4fb0-914b-c20ab1877f85 service placement] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:54.952898 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d7e1d2-73cb-4fb0-914b-c20ab1877f85 service placement] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:54.993602 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:55.012602 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fea316-db8a-4410-aa0a-805f86364a81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:55.013241 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fea316-db8a-4410-aa0a-805f86364a81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:55.013523 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fea316-db8a-4410-aa0a-805f86364a81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:55.013800 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fea316-db8a-4410-aa0a-805f86364a81 admin admin] PATH_INFO: `/v3/projects/01edf84df6ed400e8ec3282c87eb8888` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:55.014951 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fea316-db8a-4410-aa0a-805f86364a81 admin admin] RBAC: Authorizing `identity:delete_project(project_id=01edf84df6ed400e8ec3282c87eb8888)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:55.016918 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fea316-db8a-4410-aa0a-805f86364a81 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:55.062177 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-f6fea316-db8a-4410-aa0a-805f86364a81 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01edf84df6ed400e8ec3282c87eb8888', 'request_id': 'req-f6fea316-db8a-4410-aa0a-805f86364a81'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:01:55.067685 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-f6fea316-db8a-4410-aa0a-805f86364a81 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01edf84df6ed400e8ec3282c87eb8888', 'request_id': 'req-f6fea316-db8a-4410-aa0a-805f86364a81'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:01:55.073901 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-f6fea316-db8a-4410-aa0a-805f86364a81 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01edf84df6ed400e8ec3282c87eb8888', 'request_id': 'req-f6fea316-db8a-4410-aa0a-805f86364a81'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:01:55.086131 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-f6fea316-db8a-4410-aa0a-805f86364a81 admin admin] The token cache is being invalidate because project 01edf84df6ed400e8ec3282c87eb8888 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80153) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:01:55.086423 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-f6fea316-db8a-4410-aa0a-805f86364a81 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:01:55.087071 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-f6fea316-db8a-4410-aa0a-805f86364a81 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:01:55.088631 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1048/2090] 10.222.0.64 () {58 vars in 1233 bytes} [Tue Aug 2 10:01:54 2022] DELETE /identity/v3/projects/01edf84df6ed400e8ec3282c87eb8888 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:01:55.093524 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0882ee8b-b813-4381-bee5-0af9611b648e None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:55.094020 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0882ee8b-b813-4381-bee5-0af9611b648e None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:55.094120 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0882ee8b-b813-4381-bee5-0af9611b648e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:55.114147 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-0882ee8b-b813-4381-bee5-0af9611b648e None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:55.115347 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-0882ee8b-b813-4381-bee5-0af9611b648e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:55.173121 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1043/2091] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:01:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:01:55.181512 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b69ca2-797c-4c0e-8a38-e25f5cf2e723 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:55.181877 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b69ca2-797c-4c0e-8a38-e25f5cf2e723 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:55.182185 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b69ca2-797c-4c0e-8a38-e25f5cf2e723 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:55.203850 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-75b69ca2-797c-4c0e-8a38-e25f5cf2e723 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:55.205008 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-75b69ca2-797c-4c0e-8a38-e25f5cf2e723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:55.212956 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1049/2092] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:01:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:01:55.217170 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0882ee8b-b813-4381-bee5-0af9611b648e None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:55.227269 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0882ee8b-b813-4381-bee5-0af9611b648e None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:55.231754 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f06385-3f2e-4758-829b-1603317d6d8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:55.232389 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f06385-3f2e-4758-829b-1603317d6d8a admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:55.232673 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f06385-3f2e-4758-829b-1603317d6d8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:55.232947 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f06385-3f2e-4758-829b-1603317d6d8a admin admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:55.233643 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f06385-3f2e-4758-829b-1603317d6d8a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:55.233951 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f06385-3f2e-4758-829b-1603317d6d8a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:55.236113 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f06385-3f2e-4758-829b-1603317d6d8a admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:55.247432 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1044/2093] 10.222.0.64 () {58 vars in 1153 bytes} [Tue Aug 2 10:01:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:01:55.251738 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b69ca2-797c-4c0e-8a38-e25f5cf2e723 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:55.261069 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b69ca2-797c-4c0e-8a38-e25f5cf2e723 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:55.265739 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a7b74f-1438-423b-80d2-89450ef1dee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:55.266370 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a7b74f-1438-423b-80d2-89450ef1dee7 admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:55.266655 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a7b74f-1438-423b-80d2-89450ef1dee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:55.266930 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a7b74f-1438-423b-80d2-89450ef1dee7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:55.267605 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a7b74f-1438-423b-80d2-89450ef1dee7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:55.269614 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a7b74f-1438-423b-80d2-89450ef1dee7 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:55.299581 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1050/2094] 10.222.0.64 () {60 vars in 1153 bytes} [Tue Aug 2 10:01:55 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:01:55.303887 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f06385-3f2e-4758-829b-1603317d6d8a admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:55.313968 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f06385-3f2e-4758-829b-1603317d6d8a admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:55.340162 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:55.377619 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098b6633-9a98-4212-ac4b-27fd3ababe2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:55.378252 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098b6633-9a98-4212-ac4b-27fd3ababe2c admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:55.378544 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098b6633-9a98-4212-ac4b-27fd3ababe2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:55.378824 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098b6633-9a98-4212-ac4b-27fd3ababe2c admin admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:55.379537 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098b6633-9a98-4212-ac4b-27fd3ababe2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:55.381560 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098b6633-9a98-4212-ac4b-27fd3ababe2c admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:55.403658 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1045/2095] 10.222.0.64 () {60 vars in 1144 bytes} [Tue Aug 2 10:01:55 2022] POST /identity/v3/users => generated 293 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:01:55.408753 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a7b74f-1438-423b-80d2-89450ef1dee7 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:55.419171 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a7b74f-1438-423b-80d2-89450ef1dee7 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:55.424596 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153b466a-d9b7-4c73-ab26-37d27daf55a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:55.425208 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153b466a-d9b7-4c73-ab26-37d27daf55a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:55.425496 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153b466a-d9b7-4c73-ab26-37d27daf55a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:55.425771 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153b466a-d9b7-4c73-ab26-37d27daf55a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:55.426466 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153b466a-d9b7-4c73-ab26-37d27daf55a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:55.428166 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153b466a-d9b7-4c73-ab26-37d27daf55a4 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:55.443318 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1051/2096] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:01:55 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:01:55.447075 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098b6633-9a98-4212-ac4b-27fd3ababe2c admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:55.458453 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098b6633-9a98-4212-ac4b-27fd3ababe2c admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:55.463197 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68337ba6-c0f5-4cbc-817c-da47c078dba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:55.463795 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68337ba6-c0f5-4cbc-817c-da47c078dba4 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:55.464078 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68337ba6-c0f5-4cbc-817c-da47c078dba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:55.464379 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68337ba6-c0f5-4cbc-817c-da47c078dba4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:55.465100 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68337ba6-c0f5-4cbc-817c-da47c078dba4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:55.466681 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68337ba6-c0f5-4cbc-817c-da47c078dba4 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:55.477858 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1046/2097] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:01:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:01:55.481994 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153b466a-d9b7-4c73-ab26-37d27daf55a4 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:55.492129 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153b466a-d9b7-4c73-ab26-37d27daf55a4 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:55.497010 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec1e371-bb6a-4ac9-97a4-bdd30ed51595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:55.497684 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec1e371-bb6a-4ac9-97a4-bdd30ed51595 admin admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:55.497966 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec1e371-bb6a-4ac9-97a4-bdd30ed51595 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:55.498254 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec1e371-bb6a-4ac9-97a4-bdd30ed51595 admin admin] PATH_INFO: `/v3/projects/9cfde2e01ccc477cb78fbc3d1074ca4e/users/56b6ea7482db4ca2ad2d4c0e093fe7db/roles/66c1074780954f8e87ed11a5c4e8c36a` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:55.514562 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec1e371-bb6a-4ac9-97a4-bdd30ed51595 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9cfde2e01ccc477cb78fbc3d1074ca4e, user_id=56b6ea7482db4ca2ad2d4c0e093fe7db, role_id=66c1074780954f8e87ed11a5c4e8c36a)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:55.517866 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec1e371-bb6a-4ac9-97a4-bdd30ed51595 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:55.537218 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1052/2098] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 10:01:55 2022] PUT /identity/v3/projects/9cfde2e01ccc477cb78fbc3d1074ca4e/users/56b6ea7482db4ca2ad2d4c0e093fe7db/roles/66c1074780954f8e87ed11a5c4e8c36a => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:01:55.549387 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c771fb-c4cd-4b25-aec3-d805ac72b2b7 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:55.549772 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c771fb-c4cd-4b25-aec3-d805ac72b2b7 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:55.550117 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c771fb-c4cd-4b25-aec3-d805ac72b2b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:55.593073 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-f9c771fb-c4cd-4b25-aec3-d805ac72b2b7 None None] MFA Rules not processed for user `56b6ea7482db4ca2ad2d4c0e093fe7db`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:55.594347 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-f9c771fb-c4cd-4b25-aec3-d805ac72b2b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:55.661823 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1047/2099] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:01:55 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:01:55.675841 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec1e371-bb6a-4ac9-97a4-bdd30ed51595 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:55.729596 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec1e371-bb6a-4ac9-97a4-bdd30ed51595 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:55.734175 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a7cd1e-ff68-4413-b0f0-0f41dc7803e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:55.734782 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a7cd1e-ff68-4413-b0f0-0f41dc7803e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:55.735088 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a7cd1e-ff68-4413-b0f0-0f41dc7803e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:55.735364 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a7cd1e-ff68-4413-b0f0-0f41dc7803e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:55.736037 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a7cd1e-ff68-4413-b0f0-0f41dc7803e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:55.737985 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a7cd1e-ff68-4413-b0f0-0f41dc7803e1 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:55.748690 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:55.768868 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1053/2100] 10.222.0.64 () {60 vars in 1153 bytes} [Tue Aug 2 10:01:55 2022] POST /identity/v3/projects => generated 387 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:01:55.772763 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c771fb-c4cd-4b25-aec3-d805ac72b2b7 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:55.784128 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c771fb-c4cd-4b25-aec3-d805ac72b2b7 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:55.841189 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87d73fe-9d21-427b-9d65-55d2ab0ab45c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:55.841810 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87d73fe-9d21-427b-9d65-55d2ab0ab45c admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:55.842097 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87d73fe-9d21-427b-9d65-55d2ab0ab45c admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:55.842371 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87d73fe-9d21-427b-9d65-55d2ab0ab45c admin admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:55.843034 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87d73fe-9d21-427b-9d65-55d2ab0ab45c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:55.844986 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87d73fe-9d21-427b-9d65-55d2ab0ab45c admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:55.871687 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1048/2101] 10.222.0.64 () {60 vars in 1144 bytes} [Tue Aug 2 10:01:55 2022] POST /identity/v3/users => generated 300 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:01:55.876445 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a7cd1e-ff68-4413-b0f0-0f41dc7803e1 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:55.889684 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a7cd1e-ff68-4413-b0f0-0f41dc7803e1 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:55.895080 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccd5e3a-a3ba-4eb3-9281-c89a9fd80a70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:55.895554 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccd5e3a-a3ba-4eb3-9281-c89a9fd80a70 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:55.895837 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccd5e3a-a3ba-4eb3-9281-c89a9fd80a70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:55.896112 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccd5e3a-a3ba-4eb3-9281-c89a9fd80a70 admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:55.896863 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccd5e3a-a3ba-4eb3-9281-c89a9fd80a70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:55.898475 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccd5e3a-a3ba-4eb3-9281-c89a9fd80a70 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:55.911862 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1054/2102] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:01:55 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:01:55.913134 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87d73fe-9d21-427b-9d65-55d2ab0ab45c admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:55.923588 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87d73fe-9d21-427b-9d65-55d2ab0ab45c admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:55.935711 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a55db1b-2144-4e0e-837b-665db098b64f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:55.936447 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a55db1b-2144-4e0e-837b-665db098b64f admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:55.936736 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a55db1b-2144-4e0e-837b-665db098b64f admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:55.937024 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a55db1b-2144-4e0e-837b-665db098b64f admin admin] PATH_INFO: `/v3/projects/5752f632884f48ac83df78e7e2e826d1/users/944e81b2f8794070954e446f96a4d126/roles/1000cf318fa14915b8091599eaead5de` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:55.954875 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a55db1b-2144-4e0e-837b-665db098b64f admin admin] RBAC: Authorizing `identity:create_grant(project_id=5752f632884f48ac83df78e7e2e826d1, user_id=944e81b2f8794070954e446f96a4d126, role_id=1000cf318fa14915b8091599eaead5de)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:55.957593 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a55db1b-2144-4e0e-837b-665db098b64f admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:55.971322 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1049/2103] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 10:01:55 2022] PUT /identity/v3/projects/5752f632884f48ac83df78e7e2e826d1/users/944e81b2f8794070954e446f96a4d126/roles/1000cf318fa14915b8091599eaead5de => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:01:55.975906 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccd5e3a-a3ba-4eb3-9281-c89a9fd80a70 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:55.986396 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccd5e3a-a3ba-4eb3-9281-c89a9fd80a70 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:56.051751 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2800363e-1981-4313-b5a9-f8b903189b5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:56.052493 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2800363e-1981-4313-b5a9-f8b903189b5e admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:56.052855 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2800363e-1981-4313-b5a9-f8b903189b5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:56.053136 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2800363e-1981-4313-b5a9-f8b903189b5e admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:56.053838 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2800363e-1981-4313-b5a9-f8b903189b5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:56.055433 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2800363e-1981-4313-b5a9-f8b903189b5e admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:56.066494 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1055/2104] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:01:55 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:01:56.070404 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a55db1b-2144-4e0e-837b-665db098b64f admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:56.076592 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:56.084809 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a55db1b-2144-4e0e-837b-665db098b64f admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:56.090451 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c8a6d2-4a2c-4664-b27c-b3cd816bd415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:56.091132 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c8a6d2-4a2c-4664-b27c-b3cd816bd415 admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:56.091419 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c8a6d2-4a2c-4664-b27c-b3cd816bd415 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:56.091699 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c8a6d2-4a2c-4664-b27c-b3cd816bd415 admin admin] PATH_INFO: `/v3/projects/5752f632884f48ac83df78e7e2e826d1/users/944e81b2f8794070954e446f96a4d126/roles/1000cf318fa14915b8091599eaead5de` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:56.096454 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c8a6d2-4a2c-4664-b27c-b3cd816bd415 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5752f632884f48ac83df78e7e2e826d1, user_id=944e81b2f8794070954e446f96a4d126, role_id=1000cf318fa14915b8091599eaead5de)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:56.097030 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c8a6d2-4a2c-4664-b27c-b3cd816bd415 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:56.110164 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1050/2105] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 10:01:56 2022] PUT /identity/v3/projects/5752f632884f48ac83df78e7e2e826d1/users/944e81b2f8794070954e446f96a4d126/roles/1000cf318fa14915b8091599eaead5de => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:01:56.120134 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a2d224-cdbb-4621-a853-051d43f87271 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:56.120487 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a2d224-cdbb-4621-a853-051d43f87271 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:56.120865 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a2d224-cdbb-4621-a853-051d43f87271 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:56.151660 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-d9a2d224-cdbb-4621-a853-051d43f87271 None None] MFA Rules not processed for user `944e81b2f8794070954e446f96a4d126`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:56.152925 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-d9a2d224-cdbb-4621-a853-051d43f87271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:56.221710 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1056/2106] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:01:56 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:01:56.233353 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c8a6d2-4a2c-4664-b27c-b3cd816bd415 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:56.235718 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-d0c8a6d2-4a2c-4664-b27c-b3cd816bd415 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:56.291931 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c8a6d2-4a2c-4664-b27c-b3cd816bd415 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:56.295409 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100e441d-70a9-4ae2-925b-822ddd44fd33 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d7de62b9ee4039bdc79627ac7fb499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:56.296010 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100e441d-70a9-4ae2-925b-822ddd44fd33 service cinder] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:56.296322 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100e441d-70a9-4ae2-925b-822ddd44fd33 service cinder] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:56.296599 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100e441d-70a9-4ae2-925b-822ddd44fd33 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:56.306309 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100e441d-70a9-4ae2-925b-822ddd44fd33 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:56.307782 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100e441d-70a9-4ae2-925b-822ddd44fd33 service cinder] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:56.317864 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1051/2107] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:01:56 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:01:56.346527 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a2d224-cdbb-4621-a853-051d43f87271 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:56.408599 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a2d224-cdbb-4621-a853-051d43f87271 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:56.413223 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd772a4-0a28-4673-a7c0-214417c75f0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:56.413858 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd772a4-0a28-4673-a7c0-214417c75f0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:56.414146 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd772a4-0a28-4673-a7c0-214417c75f0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:56.414423 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd772a4-0a28-4673-a7c0-214417c75f0e admin admin] PATH_INFO: `/v3/users/56b6ea7482db4ca2ad2d4c0e093fe7db` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:56.415603 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd772a4-0a28-4673-a7c0-214417c75f0e admin admin] RBAC: Authorizing `identity:delete_user(user_id=56b6ea7482db4ca2ad2d4c0e093fe7db)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:56.417573 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd772a4-0a28-4673-a7c0-214417c75f0e admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:56.419825 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:56.472450 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-fbd772a4-0a28-4673-a7c0-214417c75f0e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56b6ea7482db4ca2ad2d4c0e093fe7db'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:01:56.479594 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-fbd772a4-0a28-4673-a7c0-214417c75f0e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56b6ea7482db4ca2ad2d4c0e093fe7db'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:01:56.487753 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-fbd772a4-0a28-4673-a7c0-214417c75f0e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56b6ea7482db4ca2ad2d4c0e093fe7db'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:01:56.488523 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-fbd772a4-0a28-4673-a7c0-214417c75f0e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56b6ea7482db4ca2ad2d4c0e093fe7db'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:01:56.489238 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-fbd772a4-0a28-4673-a7c0-214417c75f0e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56b6ea7482db4ca2ad2d4c0e093fe7db'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:01:56.507047 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1057/2108] 10.222.0.64 () {58 vars in 1224 bytes} [Tue Aug 2 10:01:56 2022] DELETE /identity/v3/users/56b6ea7482db4ca2ad2d4c0e093fe7db => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:01:56.511510 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7839aa9c-97a9-4b89-b87e-87a004aec248 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:56.511799 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7839aa9c-97a9-4b89-b87e-87a004aec248 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:56.512083 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7839aa9c-97a9-4b89-b87e-87a004aec248 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:56.530637 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-7839aa9c-97a9-4b89-b87e-87a004aec248 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:56.532460 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-7839aa9c-97a9-4b89-b87e-87a004aec248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:56.593311 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1052/2109] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:01:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:01:56.606493 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd772a4-0a28-4673-a7c0-214417c75f0e admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:56.607892 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-fbd772a4-0a28-4673-a7c0-214417c75f0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:56.673309 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd772a4-0a28-4673-a7c0-214417c75f0e admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:56.677121 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a819b8d-afe5-49aa-bfbe-2fc7991952a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:56.678061 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a819b8d-afe5-49aa-bfbe-2fc7991952a8 service neutron] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:56.678351 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a819b8d-afe5-49aa-bfbe-2fc7991952a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:56.678629 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a819b8d-afe5-49aa-bfbe-2fc7991952a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:56.687671 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a819b8d-afe5-49aa-bfbe-2fc7991952a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:56.689441 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a819b8d-afe5-49aa-bfbe-2fc7991952a8 service neutron] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:56.698771 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1058/2110] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:01:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:01:57.055683 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7839aa9c-97a9-4b89-b87e-87a004aec248 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:57.057130 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-7839aa9c-97a9-4b89-b87e-87a004aec248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:57.069560 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7839aa9c-97a9-4b89-b87e-87a004aec248 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:57.074182 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a64163-d644-411b-bdc8-1c842b2c87d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:57.074825 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a64163-d644-411b-bdc8-1c842b2c87d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:57.075360 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a64163-d644-411b-bdc8-1c842b2c87d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:57.075632 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a64163-d644-411b-bdc8-1c842b2c87d8 admin admin] PATH_INFO: `/v3/projects/9cfde2e01ccc477cb78fbc3d1074ca4e` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:57.076799 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a64163-d644-411b-bdc8-1c842b2c87d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9cfde2e01ccc477cb78fbc3d1074ca4e)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:57.079225 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a64163-d644-411b-bdc8-1c842b2c87d8 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:57.125269 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-37a64163-d644-411b-bdc8-1c842b2c87d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9cfde2e01ccc477cb78fbc3d1074ca4e', 'request_id': 'req-37a64163-d644-411b-bdc8-1c842b2c87d8'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:01:57.131576 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-37a64163-d644-411b-bdc8-1c842b2c87d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9cfde2e01ccc477cb78fbc3d1074ca4e', 'request_id': 'req-37a64163-d644-411b-bdc8-1c842b2c87d8'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:01:57.139314 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-37a64163-d644-411b-bdc8-1c842b2c87d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9cfde2e01ccc477cb78fbc3d1074ca4e', 'request_id': 'req-37a64163-d644-411b-bdc8-1c842b2c87d8'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:01:57.153495 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-37a64163-d644-411b-bdc8-1c842b2c87d8 admin admin] The token cache is being invalidate because project 9cfde2e01ccc477cb78fbc3d1074ca4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80154) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:01:57.153791 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-37a64163-d644-411b-bdc8-1c842b2c87d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:01:57.154442 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-37a64163-d644-411b-bdc8-1c842b2c87d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:01:57.156008 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1053/2111] 10.222.0.64 () {58 vars in 1233 bytes} [Tue Aug 2 10:01:57 2022] DELETE /identity/v3/projects/9cfde2e01ccc477cb78fbc3d1074ca4e => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:01:57.160571 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a819b8d-afe5-49aa-bfbe-2fc7991952a8 service neutron] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:57.170118 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a819b8d-afe5-49aa-bfbe-2fc7991952a8 service neutron] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:57.224952 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dacb4f-3b91-487c-8ccf-7396f60cc2ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:57.225590 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dacb4f-3b91-487c-8ccf-7396f60cc2ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:57.225871 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dacb4f-3b91-487c-8ccf-7396f60cc2ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:57.226230 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dacb4f-3b91-487c-8ccf-7396f60cc2ff admin admin] PATH_INFO: `/v3/users/944e81b2f8794070954e446f96a4d126` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:57.227379 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dacb4f-3b91-487c-8ccf-7396f60cc2ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=944e81b2f8794070954e446f96a4d126)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:57.229356 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dacb4f-3b91-487c-8ccf-7396f60cc2ff admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:57.256789 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:57.276053 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-68dacb4f-3b91-487c-8ccf-7396f60cc2ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '944e81b2f8794070954e446f96a4d126'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:01:57.283292 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-68dacb4f-3b91-487c-8ccf-7396f60cc2ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '944e81b2f8794070954e446f96a4d126'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:01:57.290502 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-68dacb4f-3b91-487c-8ccf-7396f60cc2ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '944e81b2f8794070954e446f96a4d126'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:01:57.291323 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-68dacb4f-3b91-487c-8ccf-7396f60cc2ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '944e81b2f8794070954e446f96a4d126'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:01:57.292018 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-68dacb4f-3b91-487c-8ccf-7396f60cc2ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '944e81b2f8794070954e446f96a4d126'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:01:57.309668 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1059/2112] 10.222.0.64 () {58 vars in 1224 bytes} [Tue Aug 2 10:01:57 2022] DELETE /identity/v3/users/944e81b2f8794070954e446f96a4d126 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:01:57.585081 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a64163-d644-411b-bdc8-1c842b2c87d8 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:57.586540 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-37a64163-d644-411b-bdc8-1c842b2c87d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:57.601653 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a64163-d644-411b-bdc8-1c842b2c87d8 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:57.614808 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:57.666355 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3da0689-72e0-40fb-8370-289c26186b9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:57.667013 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3da0689-72e0-40fb-8370-289c26186b9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:57.667296 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3da0689-72e0-40fb-8370-289c26186b9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:57.667572 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3da0689-72e0-40fb-8370-289c26186b9e admin admin] PATH_INFO: `/v3/projects/5752f632884f48ac83df78e7e2e826d1` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:57.669044 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3da0689-72e0-40fb-8370-289c26186b9e admin admin] RBAC: Authorizing `identity:delete_project(project_id=5752f632884f48ac83df78e7e2e826d1)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:57.670655 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3da0689-72e0-40fb-8370-289c26186b9e admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:57.734473 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-e3da0689-72e0-40fb-8370-289c26186b9e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5752f632884f48ac83df78e7e2e826d1', 'request_id': 'req-e3da0689-72e0-40fb-8370-289c26186b9e'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:01:57.739839 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-e3da0689-72e0-40fb-8370-289c26186b9e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5752f632884f48ac83df78e7e2e826d1', 'request_id': 'req-e3da0689-72e0-40fb-8370-289c26186b9e'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:01:57.746143 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-e3da0689-72e0-40fb-8370-289c26186b9e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5752f632884f48ac83df78e7e2e826d1', 'request_id': 'req-e3da0689-72e0-40fb-8370-289c26186b9e'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:01:57.761318 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-e3da0689-72e0-40fb-8370-289c26186b9e admin admin] The token cache is being invalidate because project 5752f632884f48ac83df78e7e2e826d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80154) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:01:57.761627 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-e3da0689-72e0-40fb-8370-289c26186b9e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:01:57.762333 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-e3da0689-72e0-40fb-8370-289c26186b9e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:01:57.764180 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1054/2113] 10.222.0.64 () {58 vars in 1233 bytes} [Tue Aug 2 10:01:57 2022] DELETE /identity/v3/projects/5752f632884f48ac83df78e7e2e826d1 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:01:57.769341 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128866cd-c468-4e3a-91c1-60defd57f9c6 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:57.769631 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128866cd-c468-4e3a-91c1-60defd57f9c6 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:57.769912 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128866cd-c468-4e3a-91c1-60defd57f9c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:57.789816 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-128866cd-c468-4e3a-91c1-60defd57f9c6 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:57.791070 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-128866cd-c468-4e3a-91c1-60defd57f9c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:57.850738 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1060/2114] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:01:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:01:57.858234 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b04f3b-37a3-45bb-8c3b-19d7cddcb6bf None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:57.858526 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b04f3b-37a3-45bb-8c3b-19d7cddcb6bf None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:57.858805 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b04f3b-37a3-45bb-8c3b-19d7cddcb6bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:57.880074 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-94b04f3b-37a3-45bb-8c3b-19d7cddcb6bf None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:57.882184 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-94b04f3b-37a3-45bb-8c3b-19d7cddcb6bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:57.891578 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1055/2115] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:01:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:01:57.895750 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128866cd-c468-4e3a-91c1-60defd57f9c6 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:57.905668 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128866cd-c468-4e3a-91c1-60defd57f9c6 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:57.911024 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcd3ef1-a382-43ef-a3b3-120b22f709b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:57.911627 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcd3ef1-a382-43ef-a3b3-120b22f709b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:57.911937 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcd3ef1-a382-43ef-a3b3-120b22f709b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:57.912217 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcd3ef1-a382-43ef-a3b3-120b22f709b9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:57.912945 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcd3ef1-a382-43ef-a3b3-120b22f709b9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:01:57.913253 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcd3ef1-a382-43ef-a3b3-120b22f709b9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:57.914864 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcd3ef1-a382-43ef-a3b3-120b22f709b9 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:57.927682 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1061/2116] 10.222.0.64 () {58 vars in 1153 bytes} [Tue Aug 2 10:01:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:01:57.931981 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b04f3b-37a3-45bb-8c3b-19d7cddcb6bf None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:57.941412 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b04f3b-37a3-45bb-8c3b-19d7cddcb6bf None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:57.946035 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b60562-1129-4145-bedd-e6832eea36cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:57.946674 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b60562-1129-4145-bedd-e6832eea36cd admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:57.946955 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b60562-1129-4145-bedd-e6832eea36cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:57.947230 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b60562-1129-4145-bedd-e6832eea36cd admin admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:57.947893 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b60562-1129-4145-bedd-e6832eea36cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:57.949808 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b60562-1129-4145-bedd-e6832eea36cd admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:57.978235 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1056/2117] 10.222.0.64 () {60 vars in 1153 bytes} [Tue Aug 2 10:01:57 2022] POST /identity/v3/projects => generated 387 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:01:57.982626 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcd3ef1-a382-43ef-a3b3-120b22f709b9 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:57.992863 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcd3ef1-a382-43ef-a3b3-120b22f709b9 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:58.051450 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5bb609-e878-4853-b255-b8cd5b851dde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:58.052108 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5bb609-e878-4853-b255-b8cd5b851dde admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:58.052444 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5bb609-e878-4853-b255-b8cd5b851dde admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:58.052723 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5bb609-e878-4853-b255-b8cd5b851dde admin admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:58.053397 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5bb609-e878-4853-b255-b8cd5b851dde admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:58.055399 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5bb609-e878-4853-b255-b8cd5b851dde admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:58.077085 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1062/2118] 10.222.0.64 () {60 vars in 1144 bytes} [Tue Aug 2 10:01:57 2022] POST /identity/v3/users => generated 294 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:01:58.081640 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b60562-1129-4145-bedd-e6832eea36cd admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:58.091822 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b60562-1129-4145-bedd-e6832eea36cd admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:58.097023 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f37480-b81f-46f1-821c-d2ae89e3df37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:58.097632 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f37480-b81f-46f1-821c-d2ae89e3df37 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:58.097954 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f37480-b81f-46f1-821c-d2ae89e3df37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:58.098232 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f37480-b81f-46f1-821c-d2ae89e3df37 admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:58.098933 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f37480-b81f-46f1-821c-d2ae89e3df37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:58.100558 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f37480-b81f-46f1-821c-d2ae89e3df37 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:58.110915 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1057/2119] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:01:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:01:58.114545 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5bb609-e878-4853-b255-b8cd5b851dde admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:58.125169 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5bb609-e878-4853-b255-b8cd5b851dde admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:58.129842 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc45b4eb-f44e-4a48-8874-ceae914e7068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:58.130445 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc45b4eb-f44e-4a48-8874-ceae914e7068 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:58.130730 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc45b4eb-f44e-4a48-8874-ceae914e7068 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:58.131004 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc45b4eb-f44e-4a48-8874-ceae914e7068 admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:58.131698 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc45b4eb-f44e-4a48-8874-ceae914e7068 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:58.133326 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc45b4eb-f44e-4a48-8874-ceae914e7068 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:58.145409 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1063/2120] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:01:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:01:58.149677 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f37480-b81f-46f1-821c-d2ae89e3df37 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:58.160665 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f37480-b81f-46f1-821c-d2ae89e3df37 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:58.165431 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0745011e-de04-4045-b917-586ac6787cf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:58.166166 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0745011e-de04-4045-b917-586ac6787cf1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:58.166452 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0745011e-de04-4045-b917-586ac6787cf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:58.166728 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0745011e-de04-4045-b917-586ac6787cf1 admin admin] PATH_INFO: `/v3/projects/67d6dfd6803c4b6d8f4bd6661141525d/users/b1825feba43e4c70a3933c048fbc4058/roles/66c1074780954f8e87ed11a5c4e8c36a` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:58.183429 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0745011e-de04-4045-b917-586ac6787cf1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=67d6dfd6803c4b6d8f4bd6661141525d, user_id=b1825feba43e4c70a3933c048fbc4058, role_id=66c1074780954f8e87ed11a5c4e8c36a)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:58.186106 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0745011e-de04-4045-b917-586ac6787cf1 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:58.198586 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1058/2121] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 10:01:58 2022] PUT /identity/v3/projects/67d6dfd6803c4b6d8f4bd6661141525d/users/b1825feba43e4c70a3933c048fbc4058/roles/66c1074780954f8e87ed11a5c4e8c36a => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:01:58.210259 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a52642-306f-4ea3-8580-8beb235f791c None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:58.210742 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a52642-306f-4ea3-8580-8beb235f791c None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:58.210844 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a52642-306f-4ea3-8580-8beb235f791c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:58.230257 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:58.241516 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-34a52642-306f-4ea3-8580-8beb235f791c None None] MFA Rules not processed for user `b1825feba43e4c70a3933c048fbc4058`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:58.242754 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-34a52642-306f-4ea3-8580-8beb235f791c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:58.305587 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1064/2122] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:01:58 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:01:58.310053 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0745011e-de04-4045-b917-586ac6787cf1 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:58.355154 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:58.374462 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0745011e-de04-4045-b917-586ac6787cf1 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:58.379657 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9092bc4-3ab3-47a4-a753-29d912f93a51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:58.380292 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9092bc4-3ab3-47a4-a753-29d912f93a51 admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:58.380584 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9092bc4-3ab3-47a4-a753-29d912f93a51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:58.380863 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9092bc4-3ab3-47a4-a753-29d912f93a51 admin admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:58.381542 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9092bc4-3ab3-47a4-a753-29d912f93a51 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:58.383444 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9092bc4-3ab3-47a4-a753-29d912f93a51 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:58.415714 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1059/2123] 10.222.0.64 () {60 vars in 1153 bytes} [Tue Aug 2 10:01:58 2022] POST /identity/v3/projects => generated 387 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:01:58.419668 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a52642-306f-4ea3-8580-8beb235f791c None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:58.430952 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a52642-306f-4ea3-8580-8beb235f791c None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:58.487194 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b78930-8fc0-4d5b-9806-62759c4c6bbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:58.487810 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b78930-8fc0-4d5b-9806-62759c4c6bbd admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:58.488100 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b78930-8fc0-4d5b-9806-62759c4c6bbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:58.488410 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b78930-8fc0-4d5b-9806-62759c4c6bbd admin admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:58.489075 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b78930-8fc0-4d5b-9806-62759c4c6bbd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:58.491004 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b78930-8fc0-4d5b-9806-62759c4c6bbd admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:58.511223 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1065/2124] 10.222.0.64 () {60 vars in 1144 bytes} [Tue Aug 2 10:01:58 2022] POST /identity/v3/users => generated 294 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:01:58.515236 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9092bc4-3ab3-47a4-a753-29d912f93a51 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:58.525630 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9092bc4-3ab3-47a4-a753-29d912f93a51 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:58.530225 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dffa81a-ee77-49f6-9543-5a8f08d5d503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:58.530825 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dffa81a-ee77-49f6-9543-5a8f08d5d503 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:58.531111 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dffa81a-ee77-49f6-9543-5a8f08d5d503 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:58.531389 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dffa81a-ee77-49f6-9543-5a8f08d5d503 admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:58.532147 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dffa81a-ee77-49f6-9543-5a8f08d5d503 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:58.533773 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dffa81a-ee77-49f6-9543-5a8f08d5d503 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:58.544335 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1060/2125] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:01:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:01:58.548610 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b78930-8fc0-4d5b-9806-62759c4c6bbd admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:58.558520 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b78930-8fc0-4d5b-9806-62759c4c6bbd admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:58.563717 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9cdf26-d379-4f70-9f86-50b11ebd601a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:58.564367 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9cdf26-d379-4f70-9f86-50b11ebd601a admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:58.564652 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9cdf26-d379-4f70-9f86-50b11ebd601a admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:58.564928 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9cdf26-d379-4f70-9f86-50b11ebd601a admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:58.565630 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9cdf26-d379-4f70-9f86-50b11ebd601a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:58.567250 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9cdf26-d379-4f70-9f86-50b11ebd601a admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:58.577296 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1066/2126] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:01:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:01:58.581231 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dffa81a-ee77-49f6-9543-5a8f08d5d503 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:58.591246 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dffa81a-ee77-49f6-9543-5a8f08d5d503 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:58.595879 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c652882f-bb86-435c-b272-013ecc200395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:58.596563 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c652882f-bb86-435c-b272-013ecc200395 admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:58.596849 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c652882f-bb86-435c-b272-013ecc200395 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:58.597124 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c652882f-bb86-435c-b272-013ecc200395 admin admin] PATH_INFO: `/v3/projects/4ac472548cf94823888ba9764cdc52a3/users/8bd1c415e67c44bfbe0c298ae68da342/roles/66c1074780954f8e87ed11a5c4e8c36a` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:58.611729 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c652882f-bb86-435c-b272-013ecc200395 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ac472548cf94823888ba9764cdc52a3, user_id=8bd1c415e67c44bfbe0c298ae68da342, role_id=66c1074780954f8e87ed11a5c4e8c36a)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:58.614381 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c652882f-bb86-435c-b272-013ecc200395 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:58.627444 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1061/2127] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 10:01:58 2022] PUT /identity/v3/projects/4ac472548cf94823888ba9764cdc52a3/users/8bd1c415e67c44bfbe0c298ae68da342/roles/66c1074780954f8e87ed11a5c4e8c36a => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:01:58.637516 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8371ad26-1959-4c1b-876f-668f13a1f767 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:58.637815 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8371ad26-1959-4c1b-876f-668f13a1f767 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:58.638099 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8371ad26-1959-4c1b-876f-668f13a1f767 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:58.665622 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-8371ad26-1959-4c1b-876f-668f13a1f767 None None] MFA Rules not processed for user `8bd1c415e67c44bfbe0c298ae68da342`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:58.666827 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-8371ad26-1959-4c1b-876f-668f13a1f767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:58.732017 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1067/2128] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:01:58 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:01:58.737034 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c652882f-bb86-435c-b272-013ecc200395 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:58.798399 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c652882f-bb86-435c-b272-013ecc200395 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:58.803075 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8c9f5b-1aa7-4d76-b77c-ff75dedcdd00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:58.803690 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8c9f5b-1aa7-4d76-b77c-ff75dedcdd00 admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:58.803978 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8c9f5b-1aa7-4d76-b77c-ff75dedcdd00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:58.804274 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8c9f5b-1aa7-4d76-b77c-ff75dedcdd00 admin admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:58.804961 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8c9f5b-1aa7-4d76-b77c-ff75dedcdd00 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:58.806872 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8c9f5b-1aa7-4d76-b77c-ff75dedcdd00 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:58.835482 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1062/2129] 10.222.0.64 () {60 vars in 1153 bytes} [Tue Aug 2 10:01:58 2022] POST /identity/v3/projects => generated 385 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:01:58.839582 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8371ad26-1959-4c1b-876f-668f13a1f767 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:58.850740 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8371ad26-1959-4c1b-876f-668f13a1f767 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:58.854334 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:58.914353 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d690865f-1b3e-4368-9aeb-0a21f69ebe4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:58.914983 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d690865f-1b3e-4368-9aeb-0a21f69ebe4e admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:58.915273 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d690865f-1b3e-4368-9aeb-0a21f69ebe4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:58.915554 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d690865f-1b3e-4368-9aeb-0a21f69ebe4e admin admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:58.916225 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d690865f-1b3e-4368-9aeb-0a21f69ebe4e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:58.918189 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d690865f-1b3e-4368-9aeb-0a21f69ebe4e admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:58.941763 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1068/2130] 10.222.0.64 () {60 vars in 1144 bytes} [Tue Aug 2 10:01:58 2022] POST /identity/v3/users => generated 299 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:01:58.945950 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8c9f5b-1aa7-4d76-b77c-ff75dedcdd00 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:58.957434 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8c9f5b-1aa7-4d76-b77c-ff75dedcdd00 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:58.962034 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb15109e-1504-4306-9d78-c8033a74cecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:58.962648 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb15109e-1504-4306-9d78-c8033a74cecc admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:58.962929 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb15109e-1504-4306-9d78-c8033a74cecc admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:58.963204 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb15109e-1504-4306-9d78-c8033a74cecc admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:58.963905 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb15109e-1504-4306-9d78-c8033a74cecc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:58.965575 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb15109e-1504-4306-9d78-c8033a74cecc admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:58.980672 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1063/2131] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:01:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:01:58.982998 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d690865f-1b3e-4368-9aeb-0a21f69ebe4e admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:58.992873 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d690865f-1b3e-4368-9aeb-0a21f69ebe4e admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:58.997433 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd0f14c-ba19-4c3e-9fbf-c640e7478a4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:58.998077 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd0f14c-ba19-4c3e-9fbf-c640e7478a4c admin admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:58.998359 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd0f14c-ba19-4c3e-9fbf-c640e7478a4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:58.998632 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd0f14c-ba19-4c3e-9fbf-c640e7478a4c admin admin] PATH_INFO: `/v3/projects/77027d79e1284e8a9259c244cb26954f/users/89c27778b69347128fdaea823c3089f8/roles/1000cf318fa14915b8091599eaead5de` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:59.015041 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd0f14c-ba19-4c3e-9fbf-c640e7478a4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=77027d79e1284e8a9259c244cb26954f, user_id=89c27778b69347128fdaea823c3089f8, role_id=1000cf318fa14915b8091599eaead5de)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:59.018357 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd0f14c-ba19-4c3e-9fbf-c640e7478a4c admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:59.030892 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1069/2132] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 10:01:58 2022] PUT /identity/v3/projects/77027d79e1284e8a9259c244cb26954f/users/89c27778b69347128fdaea823c3089f8/roles/1000cf318fa14915b8091599eaead5de => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:01:59.035338 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb15109e-1504-4306-9d78-c8033a74cecc admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:59.045291 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb15109e-1504-4306-9d78-c8033a74cecc admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:59.104415 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0b416a-5052-421d-a906-a3c26d685977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:59.105036 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0b416a-5052-421d-a906-a3c26d685977 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:59.105323 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0b416a-5052-421d-a906-a3c26d685977 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:59.105601 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0b416a-5052-421d-a906-a3c26d685977 admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:59.106301 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0b416a-5052-421d-a906-a3c26d685977 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:59.107913 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0b416a-5052-421d-a906-a3c26d685977 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:59.119016 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1064/2133] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:01:59 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:01:59.123221 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd0f14c-ba19-4c3e-9fbf-c640e7478a4c admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:59.133517 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd0f14c-ba19-4c3e-9fbf-c640e7478a4c admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:59.138320 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eead345c-9345-4fd8-8498-bcd0acfd1455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:59.139057 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eead345c-9345-4fd8-8498-bcd0acfd1455 admin admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:59.139415 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eead345c-9345-4fd8-8498-bcd0acfd1455 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:59.139759 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eead345c-9345-4fd8-8498-bcd0acfd1455 admin admin] PATH_INFO: `/v3/projects/77027d79e1284e8a9259c244cb26954f/users/89c27778b69347128fdaea823c3089f8/roles/1000cf318fa14915b8091599eaead5de` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:59.142263 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eead345c-9345-4fd8-8498-bcd0acfd1455 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77027d79e1284e8a9259c244cb26954f, user_id=89c27778b69347128fdaea823c3089f8, role_id=1000cf318fa14915b8091599eaead5de)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:59.145165 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eead345c-9345-4fd8-8498-bcd0acfd1455 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:59.155844 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1070/2134] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 10:01:59 2022] PUT /identity/v3/projects/77027d79e1284e8a9259c244cb26954f/users/89c27778b69347128fdaea823c3089f8/roles/1000cf318fa14915b8091599eaead5de => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:01:59.166487 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd186ebb-b519-4f01-aaac-596ebebebf3f None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:59.166984 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd186ebb-b519-4f01-aaac-596ebebebf3f None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:59.167078 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd186ebb-b519-4f01-aaac-596ebebebf3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:59.174769 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:01:59.200647 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-cd186ebb-b519-4f01-aaac-596ebebebf3f None None] MFA Rules not processed for user `89c27778b69347128fdaea823c3089f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:01:59.202029 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-cd186ebb-b519-4f01-aaac-596ebebebf3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:59.277306 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1065/2135] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:01:59 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:01:59.287941 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eead345c-9345-4fd8-8498-bcd0acfd1455 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:59.289815 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-eead345c-9345-4fd8-8498-bcd0acfd1455 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:01:59.347473 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eead345c-9345-4fd8-8498-bcd0acfd1455 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:59.351095 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df5009b-8693-4e25-813b-ba6320229271 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d7de62b9ee4039bdc79627ac7fb499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:59.351708 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df5009b-8693-4e25-813b-ba6320229271 service cinder] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:59.351994 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df5009b-8693-4e25-813b-ba6320229271 service cinder] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:59.352292 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df5009b-8693-4e25-813b-ba6320229271 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:59.361650 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df5009b-8693-4e25-813b-ba6320229271 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:59.363091 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df5009b-8693-4e25-813b-ba6320229271 service cinder] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:59.372301 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1071/2136] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:01:59 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:01:59.892444 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd186ebb-b519-4f01-aaac-596ebebebf3f None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:01:59.902478 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd186ebb-b519-4f01-aaac-596ebebebf3f None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:01:59.906037 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962de6d5-4a73-4e02-a101-b820178809dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d7de62b9ee4039bdc79627ac7fb499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:01:59.906671 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962de6d5-4a73-4e02-a101-b820178809dd service cinder] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:01:59.906952 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962de6d5-4a73-4e02-a101-b820178809dd service cinder] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:01:59.907251 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962de6d5-4a73-4e02-a101-b820178809dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:01:59.964561 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962de6d5-4a73-4e02-a101-b820178809dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:01:59.965995 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962de6d5-4a73-4e02-a101-b820178809dd service cinder] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:01:59.974855 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1066/2137] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:01:59 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:02:09.368150 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=67,keystone:INSERT=5 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:09.971220 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=61 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:19.752153 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df5009b-8693-4e25-813b-ba6320229271 service cinder] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:19.754366 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-9df5009b-8693-4e25-813b-ba6320229271 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:19.765557 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df5009b-8693-4e25-813b-ba6320229271 service cinder] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:19.769320 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b971735-43ab-4d14-97d6-bbd4450dbb77 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d7de62b9ee4039bdc79627ac7fb499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:19.769939 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b971735-43ab-4d14-97d6-bbd4450dbb77 service cinder] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:19.770221 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b971735-43ab-4d14-97d6-bbd4450dbb77 service cinder] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:19.770494 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b971735-43ab-4d14-97d6-bbd4450dbb77 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:19.826545 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b971735-43ab-4d14-97d6-bbd4450dbb77 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:19.827987 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b971735-43ab-4d14-97d6-bbd4450dbb77 service cinder] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:19.836960 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1072/2138] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:02:19 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:02:21.969461 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962de6d5-4a73-4e02-a101-b820178809dd service cinder] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:22.033472 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962de6d5-4a73-4e02-a101-b820178809dd service cinder] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:22.038892 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273aac93-f701-4800-bfdf-266dfb6db460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:22.039600 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273aac93-f701-4800-bfdf-266dfb6db460 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:22.039830 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273aac93-f701-4800-bfdf-266dfb6db460 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:22.040112 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273aac93-f701-4800-bfdf-266dfb6db460 admin admin] PATH_INFO: `/v3/users/b1825feba43e4c70a3933c048fbc4058` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:22.041341 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273aac93-f701-4800-bfdf-266dfb6db460 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1825feba43e4c70a3933c048fbc4058)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:22.043286 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273aac93-f701-4800-bfdf-266dfb6db460 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:22.098527 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-273aac93-f701-4800-bfdf-266dfb6db460 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1825feba43e4c70a3933c048fbc4058'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:02:22.105953 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-273aac93-f701-4800-bfdf-266dfb6db460 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1825feba43e4c70a3933c048fbc4058'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:02:22.106641 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-273aac93-f701-4800-bfdf-266dfb6db460 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1825feba43e4c70a3933c048fbc4058'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:02:22.116006 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-273aac93-f701-4800-bfdf-266dfb6db460 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1825feba43e4c70a3933c048fbc4058'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:02:22.132928 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-273aac93-f701-4800-bfdf-266dfb6db460 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1825feba43e4c70a3933c048fbc4058'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:02:22.139972 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1067/2139] 10.222.0.64 () {58 vars in 1224 bytes} [Tue Aug 2 10:02:21 2022] DELETE /identity/v3/users/b1825feba43e4c70a3933c048fbc4058 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:02:22.145046 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913e32d0-88bf-4588-affc-a4ae3a175e2a None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:22.145486 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913e32d0-88bf-4588-affc-a4ae3a175e2a None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:22.145579 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913e32d0-88bf-4588-affc-a4ae3a175e2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:22.164980 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-913e32d0-88bf-4588-affc-a4ae3a175e2a None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:02:22.166212 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-913e32d0-88bf-4588-affc-a4ae3a175e2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:22.229533 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1073/2140] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:02:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:02:22.240013 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273aac93-f701-4800-bfdf-266dfb6db460 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:22.240808 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913e32d0-88bf-4588-affc-a4ae3a175e2a None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:22.241738 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-273aac93-f701-4800-bfdf-266dfb6db460 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:22.242295 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-913e32d0-88bf-4588-affc-a4ae3a175e2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:22.255392 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273aac93-f701-4800-bfdf-266dfb6db460 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:22.262094 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76c8262-038f-46a1-b926-893ced08f88e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:22.262872 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76c8262-038f-46a1-b926-893ced08f88e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:22.263536 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76c8262-038f-46a1-b926-893ced08f88e admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:22.263536 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76c8262-038f-46a1-b926-893ced08f88e admin admin] PATH_INFO: `/v3/users/79dd373c5e5546a59357479b9daf304f` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:22.264760 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76c8262-038f-46a1-b926-893ced08f88e admin admin] RBAC: Authorizing `identity:delete_user(user_id=79dd373c5e5546a59357479b9daf304f)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:22.266704 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76c8262-038f-46a1-b926-893ced08f88e admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:22.324588 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-c76c8262-038f-46a1-b926-893ced08f88e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79dd373c5e5546a59357479b9daf304f'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:02:22.334635 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-c76c8262-038f-46a1-b926-893ced08f88e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79dd373c5e5546a59357479b9daf304f'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:02:22.335592 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913e32d0-88bf-4588-affc-a4ae3a175e2a None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:22.336463 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-c76c8262-038f-46a1-b926-893ced08f88e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79dd373c5e5546a59357479b9daf304f'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:02:22.341041 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b71f002-400d-45d8-9294-e2944a1a36b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:22.341929 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b71f002-400d-45d8-9294-e2944a1a36b9 service neutron] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:22.342243 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b71f002-400d-45d8-9294-e2944a1a36b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:22.342582 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b71f002-400d-45d8-9294-e2944a1a36b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:22.344999 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-0b71f002-400d-45d8-9294-e2944a1a36b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:22.345923 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-c76c8262-038f-46a1-b926-893ced08f88e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79dd373c5e5546a59357479b9daf304f'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:02:22.355710 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:22.358434 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b71f002-400d-45d8-9294-e2944a1a36b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:22.359984 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b71f002-400d-45d8-9294-e2944a1a36b9 service neutron] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:22.361662 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-c76c8262-038f-46a1-b926-893ced08f88e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79dd373c5e5546a59357479b9daf304f'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:02:22.368756 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1068/2141] 10.222.0.64 () {58 vars in 1224 bytes} [Tue Aug 2 10:02:22 2022] DELETE /identity/v3/users/79dd373c5e5546a59357479b9daf304f => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:02:22.370667 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1074/2142] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:02:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:02:22.373333 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aded9ad-0ac6-4d7f-8705-b750387f4342 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:22.373619 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aded9ad-0ac6-4d7f-8705-b750387f4342 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:22.373891 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aded9ad-0ac6-4d7f-8705-b750387f4342 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:22.393226 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-0aded9ad-0ac6-4d7f-8705-b750387f4342 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:02:22.394973 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-0aded9ad-0ac6-4d7f-8705-b750387f4342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:22.457950 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1069/2143] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:02:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:02:22.467699 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b71f002-400d-45d8-9294-e2944a1a36b9 service neutron] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:22.469699 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-0b71f002-400d-45d8-9294-e2944a1a36b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:22.481794 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b71f002-400d-45d8-9294-e2944a1a36b9 service neutron] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:22.531759 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:22.542771 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0739bb9b-c91b-4b9f-9760-a2e551705915 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:22.543460 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0739bb9b-c91b-4b9f-9760-a2e551705915 service neutron] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:22.543747 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0739bb9b-c91b-4b9f-9760-a2e551705915 service neutron] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:22.544170 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0739bb9b-c91b-4b9f-9760-a2e551705915 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:22.555519 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0739bb9b-c91b-4b9f-9760-a2e551705915 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:22.557133 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0739bb9b-c91b-4b9f-9760-a2e551705915 service neutron] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:22.570378 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1075/2144] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:02:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:02:22.822075 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aded9ad-0ac6-4d7f-8705-b750387f4342 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:22.823639 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-0aded9ad-0ac6-4d7f-8705-b750387f4342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:22.837186 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aded9ad-0ac6-4d7f-8705-b750387f4342 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:22.844508 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af63cfbe-1b40-4c61-84c0-86f360adfc0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:22.848977 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af63cfbe-1b40-4c61-84c0-86f360adfc0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:22.849520 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af63cfbe-1b40-4c61-84c0-86f360adfc0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:22.850017 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af63cfbe-1b40-4c61-84c0-86f360adfc0f admin admin] PATH_INFO: `/v3/projects/67d6dfd6803c4b6d8f4bd6661141525d` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:22.851659 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af63cfbe-1b40-4c61-84c0-86f360adfc0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=67d6dfd6803c4b6d8f4bd6661141525d)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:22.853842 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af63cfbe-1b40-4c61-84c0-86f360adfc0f admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:22.928534 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-af63cfbe-1b40-4c61-84c0-86f360adfc0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67d6dfd6803c4b6d8f4bd6661141525d', 'request_id': 'req-af63cfbe-1b40-4c61-84c0-86f360adfc0f'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:02:22.938549 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-af63cfbe-1b40-4c61-84c0-86f360adfc0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67d6dfd6803c4b6d8f4bd6661141525d', 'request_id': 'req-af63cfbe-1b40-4c61-84c0-86f360adfc0f'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:02:22.946906 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-af63cfbe-1b40-4c61-84c0-86f360adfc0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67d6dfd6803c4b6d8f4bd6661141525d', 'request_id': 'req-af63cfbe-1b40-4c61-84c0-86f360adfc0f'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:02:22.961996 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-af63cfbe-1b40-4c61-84c0-86f360adfc0f admin admin] The token cache is being invalidate because project 67d6dfd6803c4b6d8f4bd6661141525d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80154) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:02:22.961996 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-af63cfbe-1b40-4c61-84c0-86f360adfc0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:02:22.962737 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-af63cfbe-1b40-4c61-84c0-86f360adfc0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:02:22.964792 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1070/2145] 10.222.0.64 () {58 vars in 1233 bytes} [Tue Aug 2 10:02:22 2022] DELETE /identity/v3/projects/67d6dfd6803c4b6d8f4bd6661141525d => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:02:22.969230 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0739bb9b-c91b-4b9f-9760-a2e551705915 service neutron] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:23.002140 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0739bb9b-c91b-4b9f-9760-a2e551705915 service neutron] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:23.084010 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3f20cc-7c52-4667-a306-947e1e3cd9c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:23.084758 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3f20cc-7c52-4667-a306-947e1e3cd9c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:23.085029 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3f20cc-7c52-4667-a306-947e1e3cd9c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:23.085310 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3f20cc-7c52-4667-a306-947e1e3cd9c7 admin admin] PATH_INFO: `/v3/users/8bd1c415e67c44bfbe0c298ae68da342` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:23.086525 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3f20cc-7c52-4667-a306-947e1e3cd9c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8bd1c415e67c44bfbe0c298ae68da342)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:23.088510 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3f20cc-7c52-4667-a306-947e1e3cd9c7 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:23.148364 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-4b3f20cc-7c52-4667-a306-947e1e3cd9c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8bd1c415e67c44bfbe0c298ae68da342'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:02:23.166089 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af63cfbe-1b40-4c61-84c0-86f360adfc0f admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:23.166751 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-4b3f20cc-7c52-4667-a306-947e1e3cd9c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8bd1c415e67c44bfbe0c298ae68da342'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:02:23.166844 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-af63cfbe-1b40-4c61-84c0-86f360adfc0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:23.166929 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-4b3f20cc-7c52-4667-a306-947e1e3cd9c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8bd1c415e67c44bfbe0c298ae68da342'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:02:23.166929 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-4b3f20cc-7c52-4667-a306-947e1e3cd9c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8bd1c415e67c44bfbe0c298ae68da342'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:02:23.167496 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-4b3f20cc-7c52-4667-a306-947e1e3cd9c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8bd1c415e67c44bfbe0c298ae68da342'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:02:23.171440 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af63cfbe-1b40-4c61-84c0-86f360adfc0f admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:23.176637 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044e3503-759c-45a9-a928-601198a40617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:23.177275 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044e3503-759c-45a9-a928-601198a40617 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:23.177960 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044e3503-759c-45a9-a928-601198a40617 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:23.177960 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044e3503-759c-45a9-a928-601198a40617 admin admin] PATH_INFO: `/v3/projects/3328d5ef35ba4d65bc089aeb3ecd16d8` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:23.179096 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044e3503-759c-45a9-a928-601198a40617 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3328d5ef35ba4d65bc089aeb3ecd16d8)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:23.181050 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044e3503-759c-45a9-a928-601198a40617 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:23.192342 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1076/2146] 10.222.0.64 () {58 vars in 1224 bytes} [Tue Aug 2 10:02:22 2022] DELETE /identity/v3/users/8bd1c415e67c44bfbe0c298ae68da342 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:02:23.249519 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-044e3503-759c-45a9-a928-601198a40617 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3328d5ef35ba4d65bc089aeb3ecd16d8', 'request_id': 'req-044e3503-759c-45a9-a928-601198a40617'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:02:23.256200 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-044e3503-759c-45a9-a928-601198a40617 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3328d5ef35ba4d65bc089aeb3ecd16d8', 'request_id': 'req-044e3503-759c-45a9-a928-601198a40617'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:02:23.263103 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-044e3503-759c-45a9-a928-601198a40617 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3328d5ef35ba4d65bc089aeb3ecd16d8', 'request_id': 'req-044e3503-759c-45a9-a928-601198a40617'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:02:23.271626 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:23.277137 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-044e3503-759c-45a9-a928-601198a40617 admin admin] The token cache is being invalidate because project 3328d5ef35ba4d65bc089aeb3ecd16d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80154) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:02:23.277442 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-044e3503-759c-45a9-a928-601198a40617 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:02:23.278170 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-044e3503-759c-45a9-a928-601198a40617 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:02:23.280575 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1071/2147] 10.222.0.64 () {58 vars in 1233 bytes} [Tue Aug 2 10:02:23 2022] DELETE /identity/v3/projects/3328d5ef35ba4d65bc089aeb3ecd16d8 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:02:23.286138 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b018d918-fe30-4775-a1a5-87b1257be44a None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:23.286451 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b018d918-fe30-4775-a1a5-87b1257be44a None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:23.286820 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b018d918-fe30-4775-a1a5-87b1257be44a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:23.307977 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-b018d918-fe30-4775-a1a5-87b1257be44a None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:02:23.309823 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-b018d918-fe30-4775-a1a5-87b1257be44a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:23.388431 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1077/2148] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:02:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:02:23.392116 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9065169f-27fc-458d-b44b-92d8233045d1 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:23.393578 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9065169f-27fc-458d-b44b-92d8233045d1 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:23.394114 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9065169f-27fc-458d-b44b-92d8233045d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:23.417394 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-9065169f-27fc-458d-b44b-92d8233045d1 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:02:23.418601 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-9065169f-27fc-458d-b44b-92d8233045d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:23.427859 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1072/2149] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:02:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:02:23.432339 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b018d918-fe30-4775-a1a5-87b1257be44a None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:23.441175 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:23.445610 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b018d918-fe30-4775-a1a5-87b1257be44a None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:23.453293 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7e4b13-e1b3-4790-863f-48cea17fbda1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:23.453929 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7e4b13-e1b3-4790-863f-48cea17fbda1 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:23.454566 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7e4b13-e1b3-4790-863f-48cea17fbda1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:23.454864 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7e4b13-e1b3-4790-863f-48cea17fbda1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:23.456896 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7e4b13-e1b3-4790-863f-48cea17fbda1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:23.457216 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7e4b13-e1b3-4790-863f-48cea17fbda1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:23.458986 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7e4b13-e1b3-4790-863f-48cea17fbda1 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:23.472815 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1078/2150] 10.222.0.64 () {58 vars in 1153 bytes} [Tue Aug 2 10:02:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:23.476840 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9065169f-27fc-458d-b44b-92d8233045d1 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:23.486882 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9065169f-27fc-458d-b44b-92d8233045d1 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:23.494225 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14f31ff-5007-41fd-99d9-497d873ca7b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:23.494781 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14f31ff-5007-41fd-99d9-497d873ca7b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:23.495068 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14f31ff-5007-41fd-99d9-497d873ca7b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:23.495343 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14f31ff-5007-41fd-99d9-497d873ca7b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:23.496025 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14f31ff-5007-41fd-99d9-497d873ca7b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:23.498150 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14f31ff-5007-41fd-99d9-497d873ca7b7 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:23.525794 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7e4b13-e1b3-4790-863f-48cea17fbda1 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:23.527796 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-db7e4b13-e1b3-4790-863f-48cea17fbda1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:23.528311 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1073/2151] 10.222.0.64 () {60 vars in 1153 bytes} [Tue Aug 2 10:02:23 2022] POST /identity/v3/projects => generated 383 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:02:23.532020 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14f31ff-5007-41fd-99d9-497d873ca7b7 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:23.542343 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7e4b13-e1b3-4790-863f-48cea17fbda1 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:23.602039 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8eb232-85b3-4393-80c6-98ec794e60de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:23.602691 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8eb232-85b3-4393-80c6-98ec794e60de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:23.602977 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8eb232-85b3-4393-80c6-98ec794e60de admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:23.603256 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8eb232-85b3-4393-80c6-98ec794e60de admin admin] PATH_INFO: `/v3/projects/4ac472548cf94823888ba9764cdc52a3` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:23.604456 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8eb232-85b3-4393-80c6-98ec794e60de admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ac472548cf94823888ba9764cdc52a3)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:23.606207 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14f31ff-5007-41fd-99d9-497d873ca7b7 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:23.607341 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8eb232-85b3-4393-80c6-98ec794e60de admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:23.611719 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd01cfb8-c693-4455-af04-5915b09d242d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:23.612401 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd01cfb8-c693-4455-af04-5915b09d242d admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:23.612897 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd01cfb8-c693-4455-af04-5915b09d242d admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:23.613272 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd01cfb8-c693-4455-af04-5915b09d242d admin admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:23.613963 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd01cfb8-c693-4455-af04-5915b09d242d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:23.616228 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd01cfb8-c693-4455-af04-5915b09d242d admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:23.645569 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1074/2152] 10.222.0.64 () {60 vars in 1144 bytes} [Tue Aug 2 10:02:23 2022] POST /identity/v3/users => generated 292 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:02:23.649447 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd01cfb8-c693-4455-af04-5915b09d242d admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:23.660156 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd01cfb8-c693-4455-af04-5915b09d242d admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:23.673965 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-ac8eb232-85b3-4393-80c6-98ec794e60de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ac472548cf94823888ba9764cdc52a3', 'request_id': 'req-ac8eb232-85b3-4393-80c6-98ec794e60de'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:02:23.680104 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-ac8eb232-85b3-4393-80c6-98ec794e60de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ac472548cf94823888ba9764cdc52a3', 'request_id': 'req-ac8eb232-85b3-4393-80c6-98ec794e60de'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:02:23.686317 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-ac8eb232-85b3-4393-80c6-98ec794e60de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ac472548cf94823888ba9764cdc52a3', 'request_id': 'req-ac8eb232-85b3-4393-80c6-98ec794e60de'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:02:23.703514 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-ac8eb232-85b3-4393-80c6-98ec794e60de admin admin] The token cache is being invalidate because project 4ac472548cf94823888ba9764cdc52a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80153) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:02:23.703954 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-ac8eb232-85b3-4393-80c6-98ec794e60de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:02:23.704740 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-ac8eb232-85b3-4393-80c6-98ec794e60de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:02:23.707258 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1079/2153] 10.222.0.64 () {58 vars in 1233 bytes} [Tue Aug 2 10:02:23 2022] DELETE /identity/v3/projects/4ac472548cf94823888ba9764cdc52a3 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:02:23.711405 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8eb232-85b3-4393-80c6-98ec794e60de admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:23.712906 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-ac8eb232-85b3-4393-80c6-98ec794e60de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:23.733739 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9519d62c-c9c1-45a8-b473-699ab8867f1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:23.734723 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9519d62c-c9c1-45a8-b473-699ab8867f1f admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:23.735183 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9519d62c-c9c1-45a8-b473-699ab8867f1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:23.735646 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9519d62c-c9c1-45a8-b473-699ab8867f1f admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:23.737349 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9519d62c-c9c1-45a8-b473-699ab8867f1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:23.740027 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9519d62c-c9c1-45a8-b473-699ab8867f1f admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:23.755778 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1075/2154] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:02:23 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:02:23.756826 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9519d62c-c9c1-45a8-b473-699ab8867f1f admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:23.758565 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-9519d62c-c9c1-45a8-b473-699ab8867f1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:23.771845 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9519d62c-c9c1-45a8-b473-699ab8867f1f admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:23.776741 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fab418-2aa9-4fad-b233-eb036a0fd7ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:23.777365 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fab418-2aa9-4fad-b233-eb036a0fd7ae admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:23.777650 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fab418-2aa9-4fad-b233-eb036a0fd7ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:23.777937 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fab418-2aa9-4fad-b233-eb036a0fd7ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:23.778645 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fab418-2aa9-4fad-b233-eb036a0fd7ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:23.780264 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fab418-2aa9-4fad-b233-eb036a0fd7ae admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:23.792571 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1076/2155] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:02:23 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:02:23.793810 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8eb232-85b3-4393-80c6-98ec794e60de admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:23.796484 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fab418-2aa9-4fad-b233-eb036a0fd7ae admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:23.800968 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd624f1-020e-40a4-8537-524fa355f672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:23.802597 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd624f1-020e-40a4-8537-524fa355f672 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:23.803218 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd624f1-020e-40a4-8537-524fa355f672 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:23.803751 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd624f1-020e-40a4-8537-524fa355f672 admin admin] PATH_INFO: `/v3/users/89c27778b69347128fdaea823c3089f8` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:23.805882 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd624f1-020e-40a4-8537-524fa355f672 admin admin] RBAC: Authorizing `identity:delete_user(user_id=89c27778b69347128fdaea823c3089f8)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:23.808428 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd624f1-020e-40a4-8537-524fa355f672 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:23.809248 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fab418-2aa9-4fad-b233-eb036a0fd7ae admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:23.817530 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f2477e-e037-4aee-aeaf-b391998d487c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:23.818388 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f2477e-e037-4aee-aeaf-b391998d487c admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:23.818788 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f2477e-e037-4aee-aeaf-b391998d487c admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:23.819166 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f2477e-e037-4aee-aeaf-b391998d487c admin admin] PATH_INFO: `/v3/projects/52ea66f641ea4e1f877d88f5ddc4c226/users/3f9506226c4846628d32aadb0f6b86be/roles/66c1074780954f8e87ed11a5c4e8c36a` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:23.841360 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f2477e-e037-4aee-aeaf-b391998d487c admin admin] RBAC: Authorizing `identity:create_grant(project_id=52ea66f641ea4e1f877d88f5ddc4c226, user_id=3f9506226c4846628d32aadb0f6b86be, role_id=66c1074780954f8e87ed11a5c4e8c36a)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:23.842154 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:23.845693 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f2477e-e037-4aee-aeaf-b391998d487c admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:23.866093 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1077/2156] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 10:02:23 2022] PUT /identity/v3/projects/52ea66f641ea4e1f877d88f5ddc4c226/users/3f9506226c4846628d32aadb0f6b86be/roles/66c1074780954f8e87ed11a5c4e8c36a => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:02:23.873085 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc1cf17-28d7-47a0-a719-4279ae0fb782 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:23.873380 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc1cf17-28d7-47a0-a719-4279ae0fb782 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:23.873672 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc1cf17-28d7-47a0-a719-4279ae0fb782 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:23.883522 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-5fd624f1-020e-40a4-8537-524fa355f672 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89c27778b69347128fdaea823c3089f8'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:02:23.892099 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-5fd624f1-020e-40a4-8537-524fa355f672 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89c27778b69347128fdaea823c3089f8'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:02:23.901785 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-6fc1cf17-28d7-47a0-a719-4279ae0fb782 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:02:23.902150 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-5fd624f1-020e-40a4-8537-524fa355f672 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89c27778b69347128fdaea823c3089f8'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:02:23.902955 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-5fd624f1-020e-40a4-8537-524fa355f672 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89c27778b69347128fdaea823c3089f8'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:02:23.903342 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-6fc1cf17-28d7-47a0-a719-4279ae0fb782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:23.903642 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-5fd624f1-020e-40a4-8537-524fa355f672 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89c27778b69347128fdaea823c3089f8'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:02:23.919946 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:23.928080 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1080/2157] 10.222.0.64 () {58 vars in 1224 bytes} [Tue Aug 2 10:02:23 2022] DELETE /identity/v3/users/89c27778b69347128fdaea823c3089f8 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:02:23.983024 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1078/2158] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:02:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:02:23.993764 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd624f1-020e-40a4-8537-524fa355f672 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:23.995391 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-5fd624f1-020e-40a4-8537-524fa355f672 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:24.074044 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd624f1-020e-40a4-8537-524fa355f672 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:24.078255 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967794d3-b086-400c-9a30-2896335864ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:24.078872 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967794d3-b086-400c-9a30-2896335864ab service neutron] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:24.079155 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967794d3-b086-400c-9a30-2896335864ab service neutron] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:24.079428 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967794d3-b086-400c-9a30-2896335864ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:24.152618 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967794d3-b086-400c-9a30-2896335864ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:24.154075 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967794d3-b086-400c-9a30-2896335864ab service neutron] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:24.164674 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1081/2159] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:02:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:02:24.248836 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc1cf17-28d7-47a0-a719-4279ae0fb782 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:24.250830 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-6fc1cf17-28d7-47a0-a719-4279ae0fb782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:24.276703 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc1cf17-28d7-47a0-a719-4279ae0fb782 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:24.283354 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0238830c-a9c6-4267-8739-9ed97c735948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:24.284190 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0238830c-a9c6-4267-8739-9ed97c735948 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:24.284657 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0238830c-a9c6-4267-8739-9ed97c735948 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:24.285077 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0238830c-a9c6-4267-8739-9ed97c735948 admin admin] PATH_INFO: `/v3/projects/77027d79e1284e8a9259c244cb26954f` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:24.286720 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0238830c-a9c6-4267-8739-9ed97c735948 admin admin] RBAC: Authorizing `identity:delete_project(project_id=77027d79e1284e8a9259c244cb26954f)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:24.289382 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0238830c-a9c6-4267-8739-9ed97c735948 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:24.347808 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-0238830c-a9c6-4267-8739-9ed97c735948 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77027d79e1284e8a9259c244cb26954f', 'request_id': 'req-0238830c-a9c6-4267-8739-9ed97c735948'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:02:24.353519 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-0238830c-a9c6-4267-8739-9ed97c735948 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77027d79e1284e8a9259c244cb26954f', 'request_id': 'req-0238830c-a9c6-4267-8739-9ed97c735948'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:02:24.360688 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-0238830c-a9c6-4267-8739-9ed97c735948 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77027d79e1284e8a9259c244cb26954f', 'request_id': 'req-0238830c-a9c6-4267-8739-9ed97c735948'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:02:24.379514 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-0238830c-a9c6-4267-8739-9ed97c735948 admin admin] The token cache is being invalidate because project 77027d79e1284e8a9259c244cb26954f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80154) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:02:24.379997 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-0238830c-a9c6-4267-8739-9ed97c735948 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:02:24.380771 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-0238830c-a9c6-4267-8739-9ed97c735948 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:02:24.382944 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1079/2160] 10.222.0.64 () {58 vars in 1233 bytes} [Tue Aug 2 10:02:24 2022] DELETE /identity/v3/projects/77027d79e1284e8a9259c244cb26954f => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:02:24.389057 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4c9c6d-5a5f-4788-a26f-f9af82c115b0 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:24.389347 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4c9c6d-5a5f-4788-a26f-f9af82c115b0 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:24.389625 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4c9c6d-5a5f-4788-a26f-f9af82c115b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:24.421927 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-6c4c9c6d-5a5f-4788-a26f-f9af82c115b0 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:02:24.423154 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-6c4c9c6d-5a5f-4788-a26f-f9af82c115b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:24.473396 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:24.501339 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1082/2161] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:02:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:02:24.501524 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c6c376-14ef-4028-9b5d-41c7b438f659 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:24.501857 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c6c376-14ef-4028-9b5d-41c7b438f659 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:24.502176 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c6c376-14ef-4028-9b5d-41c7b438f659 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:24.524961 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-48c6c376-14ef-4028-9b5d-41c7b438f659 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:02:24.526318 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-48c6c376-14ef-4028-9b5d-41c7b438f659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:24.543746 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4c9c6d-5a5f-4788-a26f-f9af82c115b0 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:24.544635 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1080/2162] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:02:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:02:24.555089 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4c9c6d-5a5f-4788-a26f-f9af82c115b0 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:24.559761 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b854e9-921a-477e-96be-5a9915637363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:24.560424 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b854e9-921a-477e-96be-5a9915637363 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:24.560706 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b854e9-921a-477e-96be-5a9915637363 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:24.560981 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b854e9-921a-477e-96be-5a9915637363 admin admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:24.561682 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b854e9-921a-477e-96be-5a9915637363 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:24.561988 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b854e9-921a-477e-96be-5a9915637363 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:24.563621 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b854e9-921a-477e-96be-5a9915637363 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:24.577609 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1083/2163] 10.222.0.64 () {58 vars in 1153 bytes} [Tue Aug 2 10:02:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:02:24.581917 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c6c376-14ef-4028-9b5d-41c7b438f659 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:24.591467 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c6c376-14ef-4028-9b5d-41c7b438f659 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:24.597685 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986c553b-dd2b-458b-ada2-ded2773cd38c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:24.598311 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986c553b-dd2b-458b-ada2-ded2773cd38c admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:24.598774 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986c553b-dd2b-458b-ada2-ded2773cd38c admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:24.599055 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986c553b-dd2b-458b-ada2-ded2773cd38c admin admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:24.599897 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986c553b-dd2b-458b-ada2-ded2773cd38c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:24.601920 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986c553b-dd2b-458b-ada2-ded2773cd38c admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:24.636722 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1081/2164] 10.222.0.64 () {60 vars in 1153 bytes} [Tue Aug 2 10:02:24 2022] POST /identity/v3/projects => generated 393 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:02:24.641424 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b854e9-921a-477e-96be-5a9915637363 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:24.653778 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b854e9-921a-477e-96be-5a9915637363 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:24.744667 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec2b93b-ceed-44b6-bbfa-b09a7ddfb79e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:24.746232 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec2b93b-ceed-44b6-bbfa-b09a7ddfb79e admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:24.748003 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec2b93b-ceed-44b6-bbfa-b09a7ddfb79e admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:24.748981 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec2b93b-ceed-44b6-bbfa-b09a7ddfb79e admin admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:24.751526 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec2b93b-ceed-44b6-bbfa-b09a7ddfb79e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:24.755339 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec2b93b-ceed-44b6-bbfa-b09a7ddfb79e admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:24.796071 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1084/2165] 10.222.0.64 () {60 vars in 1144 bytes} [Tue Aug 2 10:02:24 2022] POST /identity/v3/users => generated 297 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:02:24.797620 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986c553b-dd2b-458b-ada2-ded2773cd38c admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:24.810579 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986c553b-dd2b-458b-ada2-ded2773cd38c admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:24.818424 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1807ad5-0651-4ea6-b9c2-4460a7c824c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:24.819093 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1807ad5-0651-4ea6-b9c2-4460a7c824c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:24.819387 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1807ad5-0651-4ea6-b9c2-4460a7c824c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:24.819705 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1807ad5-0651-4ea6-b9c2-4460a7c824c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:24.820475 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1807ad5-0651-4ea6-b9c2-4460a7c824c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:24.822222 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1807ad5-0651-4ea6-b9c2-4460a7c824c0 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:24.841049 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1082/2166] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:02:24 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:02:24.849910 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec2b93b-ceed-44b6-bbfa-b09a7ddfb79e admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:24.865563 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec2b93b-ceed-44b6-bbfa-b09a7ddfb79e admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:24.872035 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82dba19-38e3-4a1d-aa29-8d6c3fef7baa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:24.872941 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82dba19-38e3-4a1d-aa29-8d6c3fef7baa admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:24.873405 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82dba19-38e3-4a1d-aa29-8d6c3fef7baa admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:24.873840 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82dba19-38e3-4a1d-aa29-8d6c3fef7baa admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:24.874804 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82dba19-38e3-4a1d-aa29-8d6c3fef7baa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:24.876639 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82dba19-38e3-4a1d-aa29-8d6c3fef7baa admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:24.897142 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1085/2167] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:02:24 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:02:24.901411 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1807ad5-0651-4ea6-b9c2-4460a7c824c0 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:24.912893 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1807ad5-0651-4ea6-b9c2-4460a7c824c0 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:24.918592 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb412808-d93c-427c-9f77-de5f914dfb45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:24.919419 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb412808-d93c-427c-9f77-de5f914dfb45 admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:24.919807 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb412808-d93c-427c-9f77-de5f914dfb45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:24.920161 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb412808-d93c-427c-9f77-de5f914dfb45 admin admin] PATH_INFO: `/v3/projects/db7d670d431242eaa6ed5049d454f18a/users/0c9a9824b42d4b999d1f8f0a2b573b36/roles/66c1074780954f8e87ed11a5c4e8c36a` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:24.938186 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb412808-d93c-427c-9f77-de5f914dfb45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db7d670d431242eaa6ed5049d454f18a, user_id=0c9a9824b42d4b999d1f8f0a2b573b36, role_id=66c1074780954f8e87ed11a5c4e8c36a)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:24.940863 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb412808-d93c-427c-9f77-de5f914dfb45 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:24.962060 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1083/2168] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 10:02:24 2022] PUT /identity/v3/projects/db7d670d431242eaa6ed5049d454f18a/users/0c9a9824b42d4b999d1f8f0a2b573b36/roles/66c1074780954f8e87ed11a5c4e8c36a => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:02:24.973479 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bb3a1f-a84c-4afb-97be-461d6c038249 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:24.973777 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bb3a1f-a84c-4afb-97be-461d6c038249 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:24.974060 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bb3a1f-a84c-4afb-97be-461d6c038249 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:25.021626 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-c4bb3a1f-a84c-4afb-97be-461d6c038249 None None] MFA Rules not processed for user `0c9a9824b42d4b999d1f8f0a2b573b36`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:02:25.022409 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-c4bb3a1f-a84c-4afb-97be-461d6c038249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:25.115611 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1086/2169] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:02:24 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:02:25.120296 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb412808-d93c-427c-9f77-de5f914dfb45 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:25.177176 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:25.205388 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb412808-d93c-427c-9f77-de5f914dfb45 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:25.211524 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53466986-e539-482c-9e41-b877d093835b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:25.212378 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53466986-e539-482c-9e41-b877d093835b admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:25.212778 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53466986-e539-482c-9e41-b877d093835b admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:25.213199 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53466986-e539-482c-9e41-b877d093835b admin admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:25.214165 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53466986-e539-482c-9e41-b877d093835b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:25.216366 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53466986-e539-482c-9e41-b877d093835b admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:25.268646 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1084/2170] 10.222.0.64 () {60 vars in 1153 bytes} [Tue Aug 2 10:02:25 2022] POST /identity/v3/projects => generated 391 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:02:25.272218 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bb3a1f-a84c-4afb-97be-461d6c038249 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:25.293128 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bb3a1f-a84c-4afb-97be-461d6c038249 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:25.358823 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:25.384383 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d961f40-8f96-4156-bcc1-0fa6d36409e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:25.385411 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d961f40-8f96-4156-bcc1-0fa6d36409e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:25.385708 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d961f40-8f96-4156-bcc1-0fa6d36409e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:25.385990 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d961f40-8f96-4156-bcc1-0fa6d36409e2 admin admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:25.386722 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d961f40-8f96-4156-bcc1-0fa6d36409e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:25.388889 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d961f40-8f96-4156-bcc1-0fa6d36409e2 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:25.417102 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1087/2171] 10.222.0.64 () {60 vars in 1144 bytes} [Tue Aug 2 10:02:25 2022] POST /identity/v3/users => generated 302 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:02:25.423008 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53466986-e539-482c-9e41-b877d093835b admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:25.457458 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53466986-e539-482c-9e41-b877d093835b admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:25.479231 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331c1c19-84f2-468c-9e51-204f8bad9f20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:25.481395 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331c1c19-84f2-468c-9e51-204f8bad9f20 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:25.481851 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331c1c19-84f2-468c-9e51-204f8bad9f20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:25.482223 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331c1c19-84f2-468c-9e51-204f8bad9f20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:25.483063 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331c1c19-84f2-468c-9e51-204f8bad9f20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:25.487401 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331c1c19-84f2-468c-9e51-204f8bad9f20 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:25.525840 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1085/2172] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:02:25 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:02:25.528516 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d961f40-8f96-4156-bcc1-0fa6d36409e2 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:25.540621 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d961f40-8f96-4156-bcc1-0fa6d36409e2 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:25.547052 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bed873-9f15-4a1a-9feb-f1fc1ec76eaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:25.547744 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bed873-9f15-4a1a-9feb-f1fc1ec76eaa admin admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:25.548033 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bed873-9f15-4a1a-9feb-f1fc1ec76eaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:25.548348 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bed873-9f15-4a1a-9feb-f1fc1ec76eaa admin admin] PATH_INFO: `/v3/projects/480f236591c14b3a9aaa322549bf1115/users/3832be58608243298834b7d7a28aaa30/roles/1000cf318fa14915b8091599eaead5de` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:25.566784 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bed873-9f15-4a1a-9feb-f1fc1ec76eaa admin admin] RBAC: Authorizing `identity:create_grant(project_id=480f236591c14b3a9aaa322549bf1115, user_id=3832be58608243298834b7d7a28aaa30, role_id=1000cf318fa14915b8091599eaead5de)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:25.569519 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bed873-9f15-4a1a-9feb-f1fc1ec76eaa admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:25.585827 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1088/2173] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 10:02:25 2022] PUT /identity/v3/projects/480f236591c14b3a9aaa322549bf1115/users/3832be58608243298834b7d7a28aaa30/roles/1000cf318fa14915b8091599eaead5de => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:02:25.589463 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331c1c19-84f2-468c-9e51-204f8bad9f20 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:25.603434 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331c1c19-84f2-468c-9e51-204f8bad9f20 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:25.671079 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1754e14e-3ee4-4d8a-857b-9772e2eb40a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:25.672026 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1754e14e-3ee4-4d8a-857b-9772e2eb40a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:25.672464 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1754e14e-3ee4-4d8a-857b-9772e2eb40a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:25.672803 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1754e14e-3ee4-4d8a-857b-9772e2eb40a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:25.673771 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1754e14e-3ee4-4d8a-857b-9772e2eb40a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:25.675545 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1754e14e-3ee4-4d8a-857b-9772e2eb40a4 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:25.686244 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1086/2174] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:02:25 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:02:25.690981 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bed873-9f15-4a1a-9feb-f1fc1ec76eaa admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:25.703850 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bed873-9f15-4a1a-9feb-f1fc1ec76eaa admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:25.710304 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6db719f-e849-4bf7-854f-dc9e67bd36e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:25.711844 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6db719f-e849-4bf7-854f-dc9e67bd36e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:25.712321 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6db719f-e849-4bf7-854f-dc9e67bd36e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:25.712611 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6db719f-e849-4bf7-854f-dc9e67bd36e8 admin admin] PATH_INFO: `/v3/projects/480f236591c14b3a9aaa322549bf1115/users/3832be58608243298834b7d7a28aaa30/roles/1000cf318fa14915b8091599eaead5de` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:25.714713 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6db719f-e849-4bf7-854f-dc9e67bd36e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=480f236591c14b3a9aaa322549bf1115, user_id=3832be58608243298834b7d7a28aaa30, role_id=1000cf318fa14915b8091599eaead5de)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:25.717463 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6db719f-e849-4bf7-854f-dc9e67bd36e8 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:25.735922 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1089/2175] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 10:02:25 2022] PUT /identity/v3/projects/480f236591c14b3a9aaa322549bf1115/users/3832be58608243298834b7d7a28aaa30/roles/1000cf318fa14915b8091599eaead5de => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:02:25.745923 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ae9322-7418-4ec6-8365-b02b46a09592 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:25.746219 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ae9322-7418-4ec6-8365-b02b46a09592 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:25.746500 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ae9322-7418-4ec6-8365-b02b46a09592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:25.780878 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-d3ae9322-7418-4ec6-8365-b02b46a09592 None None] MFA Rules not processed for user `3832be58608243298834b7d7a28aaa30`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:02:25.782449 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-d3ae9322-7418-4ec6-8365-b02b46a09592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:25.879749 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1087/2176] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:02:25 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:02:25.898631 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6db719f-e849-4bf7-854f-dc9e67bd36e8 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:25.900668 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-c6db719f-e849-4bf7-854f-dc9e67bd36e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:25.977041 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6db719f-e849-4bf7-854f-dc9e67bd36e8 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:25.981843 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c168198-e5b9-422e-94b7-a3617da1ea38 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d7de62b9ee4039bdc79627ac7fb499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:25.982481 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c168198-e5b9-422e-94b7-a3617da1ea38 service cinder] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:25.982839 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c168198-e5b9-422e-94b7-a3617da1ea38 service cinder] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:25.983173 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c168198-e5b9-422e-94b7-a3617da1ea38 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:25.995494 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c168198-e5b9-422e-94b7-a3617da1ea38 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:25.997046 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c168198-e5b9-422e-94b7-a3617da1ea38 service cinder] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:26.010817 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1090/2177] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:02:25 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:02:26.061968 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ae9322-7418-4ec6-8365-b02b46a09592 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:26.065146 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-d3ae9322-7418-4ec6-8365-b02b46a09592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:26.078457 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ae9322-7418-4ec6-8365-b02b46a09592 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:26.083484 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d03b710-cc07-409a-99b4-0cbca5c86047 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d7de62b9ee4039bdc79627ac7fb499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:26.084371 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d03b710-cc07-409a-99b4-0cbca5c86047 service cinder] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:26.084717 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d03b710-cc07-409a-99b4-0cbca5c86047 service cinder] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:26.085059 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d03b710-cc07-409a-99b4-0cbca5c86047 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:26.130444 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:26.192178 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d03b710-cc07-409a-99b4-0cbca5c86047 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:26.193620 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d03b710-cc07-409a-99b4-0cbca5c86047 service cinder] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:26.203144 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1088/2178] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:02:26 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:02:28.592184 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb449b7-014a-41da-8f57-e383a24dad6b None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:28.592825 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb449b7-014a-41da-8f57-e383a24dad6b None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:28.592825 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb449b7-014a-41da-8f57-e383a24dad6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:28.619812 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-ceb449b7-014a-41da-8f57-e383a24dad6b None None] MFA Rules not processed for user `7739a00c6524467d8fd139183c6667cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:02:28.621275 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-ceb449b7-014a-41da-8f57-e383a24dad6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:28.713896 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1091/2179] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:02:28 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:02:30.001252 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac52870-bb5a-4582-92fb-9b1fb02511bf None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:30.002199 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac52870-bb5a-4582-92fb-9b1fb02511bf None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:30.002511 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac52870-bb5a-4582-92fb-9b1fb02511bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:30.088573 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-eac52870-bb5a-4582-92fb-9b1fb02511bf None None] MFA Rules not processed for user `3f9506226c4846628d32aadb0f6b86be`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:02:30.090325 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-eac52870-bb5a-4582-92fb-9b1fb02511bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:30.265546 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1089/2180] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:02:29 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:02:30.267274 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb449b7-014a-41da-8f57-e383a24dad6b None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:30.269903 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-ceb449b7-014a-41da-8f57-e383a24dad6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:30.366681 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:30.399469 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb449b7-014a-41da-8f57-e383a24dad6b None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:30.407992 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960b6ded-67a2-4c1f-babf-2731784f5fae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:30.409585 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960b6ded-67a2-4c1f-babf-2731784f5fae admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:30.409749 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960b6ded-67a2-4c1f-babf-2731784f5fae admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:30.410027 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960b6ded-67a2-4c1f-babf-2731784f5fae admin admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:30.410716 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960b6ded-67a2-4c1f-babf-2731784f5fae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:30.413213 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960b6ded-67a2-4c1f-babf-2731784f5fae admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:30.463880 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac52870-bb5a-4582-92fb-9b1fb02511bf None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:30.466965 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1092/2181] 10.222.0.64 () {60 vars in 1153 bytes} [Tue Aug 2 10:02:30 2022] POST /identity/v3/projects => generated 385 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:02:30.483917 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac52870-bb5a-4582-92fb-9b1fb02511bf None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:30.600790 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb60fbc-fa76-4b72-b6b2-0945c88842b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:30.601590 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb60fbc-fa76-4b72-b6b2-0945c88842b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:30.601968 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb60fbc-fa76-4b72-b6b2-0945c88842b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:30.602324 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb60fbc-fa76-4b72-b6b2-0945c88842b8 admin admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:30.603167 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb60fbc-fa76-4b72-b6b2-0945c88842b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:30.605600 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb60fbc-fa76-4b72-b6b2-0945c88842b8 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:30.676292 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1090/2182] 10.222.0.64 () {60 vars in 1144 bytes} [Tue Aug 2 10:02:30 2022] POST /identity/v3/users => generated 299 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:02:30.680374 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960b6ded-67a2-4c1f-babf-2731784f5fae admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:30.722660 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960b6ded-67a2-4c1f-babf-2731784f5fae admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:30.737039 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973a7d81-cae2-4f86-8b29-bf3cc5ea5579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:30.737401 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973a7d81-cae2-4f86-8b29-bf3cc5ea5579 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:30.737762 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973a7d81-cae2-4f86-8b29-bf3cc5ea5579 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:30.737964 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973a7d81-cae2-4f86-8b29-bf3cc5ea5579 admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:30.738681 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973a7d81-cae2-4f86-8b29-bf3cc5ea5579 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:30.748474 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973a7d81-cae2-4f86-8b29-bf3cc5ea5579 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:30.775119 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1093/2183] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:02:30 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:02:30.776187 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb60fbc-fa76-4b72-b6b2-0945c88842b8 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:30.831999 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb60fbc-fa76-4b72-b6b2-0945c88842b8 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:30.857433 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a0bdc4-9c87-4288-b903-a6ec4b46cb38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:30.858705 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a0bdc4-9c87-4288-b903-a6ec4b46cb38 admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:30.859277 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a0bdc4-9c87-4288-b903-a6ec4b46cb38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:30.860145 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a0bdc4-9c87-4288-b903-a6ec4b46cb38 admin admin] PATH_INFO: `/v3/projects/e88d8c3ba87e4a4db25439b1f266fe23/users/8026b988ac8d4342b0d3fd2ccfae915c/roles/1000cf318fa14915b8091599eaead5de` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:30.911977 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a0bdc4-9c87-4288-b903-a6ec4b46cb38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e88d8c3ba87e4a4db25439b1f266fe23, user_id=8026b988ac8d4342b0d3fd2ccfae915c, role_id=1000cf318fa14915b8091599eaead5de)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:30.914714 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a0bdc4-9c87-4288-b903-a6ec4b46cb38 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:30.944074 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973a7d81-cae2-4f86-8b29-bf3cc5ea5579 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:30.945517 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1091/2184] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 10:02:30 2022] PUT /identity/v3/projects/e88d8c3ba87e4a4db25439b1f266fe23/users/8026b988ac8d4342b0d3fd2ccfae915c/roles/1000cf318fa14915b8091599eaead5de => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:02:30.961888 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973a7d81-cae2-4f86-8b29-bf3cc5ea5579 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:31.069488 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f59bce-d33e-4db4-8e39-ebdfd57bd6c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:31.070395 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f59bce-d33e-4db4-8e39-ebdfd57bd6c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:31.071084 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f59bce-d33e-4db4-8e39-ebdfd57bd6c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:31.071475 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f59bce-d33e-4db4-8e39-ebdfd57bd6c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:31.072439 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f59bce-d33e-4db4-8e39-ebdfd57bd6c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:31.074571 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f59bce-d33e-4db4-8e39-ebdfd57bd6c4 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:31.092336 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1094/2185] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:02:30 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:02:31.096365 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a0bdc4-9c87-4288-b903-a6ec4b46cb38 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:31.118586 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a0bdc4-9c87-4288-b903-a6ec4b46cb38 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:31.125864 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b30c4b0-a8dd-40fb-999f-b86c567cd4f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:31.126633 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b30c4b0-a8dd-40fb-999f-b86c567cd4f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:31.127019 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b30c4b0-a8dd-40fb-999f-b86c567cd4f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:31.127944 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b30c4b0-a8dd-40fb-999f-b86c567cd4f1 admin admin] PATH_INFO: `/v3/projects/e88d8c3ba87e4a4db25439b1f266fe23/users/8026b988ac8d4342b0d3fd2ccfae915c/roles/1000cf318fa14915b8091599eaead5de` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:31.130131 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b30c4b0-a8dd-40fb-999f-b86c567cd4f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e88d8c3ba87e4a4db25439b1f266fe23, user_id=8026b988ac8d4342b0d3fd2ccfae915c, role_id=1000cf318fa14915b8091599eaead5de)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:31.133076 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b30c4b0-a8dd-40fb-999f-b86c567cd4f1 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:31.149011 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1092/2186] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 10:02:31 2022] PUT /identity/v3/projects/e88d8c3ba87e4a4db25439b1f266fe23/users/8026b988ac8d4342b0d3fd2ccfae915c/roles/1000cf318fa14915b8091599eaead5de => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:02:31.373435 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f59bce-d33e-4db4-8e39-ebdfd57bd6c4 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:31.384316 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f59bce-d33e-4db4-8e39-ebdfd57bd6c4 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:31.455502 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c961ec9-2cb2-4d15-bc31-56211154c4ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:31.456312 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c961ec9-2cb2-4d15-bc31-56211154c4ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:31.456665 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c961ec9-2cb2-4d15-bc31-56211154c4ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:31.457007 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c961ec9-2cb2-4d15-bc31-56211154c4ef admin admin] PATH_INFO: `/v3/users/0c9a9824b42d4b999d1f8f0a2b573b36` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:31.458299 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c961ec9-2cb2-4d15-bc31-56211154c4ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c9a9824b42d4b999d1f8f0a2b573b36)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:31.461958 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c961ec9-2cb2-4d15-bc31-56211154c4ef admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:31.518825 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-5c961ec9-2cb2-4d15-bc31-56211154c4ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c9a9824b42d4b999d1f8f0a2b573b36'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:02:31.527844 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-5c961ec9-2cb2-4d15-bc31-56211154c4ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c9a9824b42d4b999d1f8f0a2b573b36'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:02:31.531097 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:31.538832 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-5c961ec9-2cb2-4d15-bc31-56211154c4ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c9a9824b42d4b999d1f8f0a2b573b36'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:02:31.539662 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-5c961ec9-2cb2-4d15-bc31-56211154c4ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c9a9824b42d4b999d1f8f0a2b573b36'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:02:31.541031 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-5c961ec9-2cb2-4d15-bc31-56211154c4ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c9a9824b42d4b999d1f8f0a2b573b36'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:02:31.566718 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1095/2187] 10.222.0.64 () {58 vars in 1224 bytes} [Tue Aug 2 10:02:31 2022] DELETE /identity/v3/users/0c9a9824b42d4b999d1f8f0a2b573b36 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:02:31.567735 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b074c9c7-5a58-4800-bd5b-ff0f03ec6852 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:31.568373 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b074c9c7-5a58-4800-bd5b-ff0f03ec6852 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:31.568772 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b074c9c7-5a58-4800-bd5b-ff0f03ec6852 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:31.595186 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-b074c9c7-5a58-4800-bd5b-ff0f03ec6852 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:02:31.598878 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-b074c9c7-5a58-4800-bd5b-ff0f03ec6852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:31.610371 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:31.677801 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1093/2188] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:02:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:02:31.687897 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c961ec9-2cb2-4d15-bc31-56211154c4ef admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:31.689495 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-5c961ec9-2cb2-4d15-bc31-56211154c4ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:31.755319 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c961ec9-2cb2-4d15-bc31-56211154c4ef admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:31.759732 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108068ad-e35c-474d-ac9b-1439294e6163 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:31.760512 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108068ad-e35c-474d-ac9b-1439294e6163 service neutron] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:31.761185 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108068ad-e35c-474d-ac9b-1439294e6163 service neutron] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:31.761465 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108068ad-e35c-474d-ac9b-1439294e6163 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:31.772547 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108068ad-e35c-474d-ac9b-1439294e6163 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:31.774352 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108068ad-e35c-474d-ac9b-1439294e6163 service neutron] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:31.787328 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1096/2189] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:02:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:02:32.191256 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b074c9c7-5a58-4800-bd5b-ff0f03ec6852 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:32.192792 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-b074c9c7-5a58-4800-bd5b-ff0f03ec6852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:32.217705 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b074c9c7-5a58-4800-bd5b-ff0f03ec6852 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:32.224432 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954c29d6-c677-4547-ac34-3214cb0abf9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:32.225098 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954c29d6-c677-4547-ac34-3214cb0abf9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:32.225381 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954c29d6-c677-4547-ac34-3214cb0abf9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:32.226173 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954c29d6-c677-4547-ac34-3214cb0abf9d admin admin] PATH_INFO: `/v3/projects/db7d670d431242eaa6ed5049d454f18a` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:32.227965 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954c29d6-c677-4547-ac34-3214cb0abf9d admin admin] RBAC: Authorizing `identity:delete_project(project_id=db7d670d431242eaa6ed5049d454f18a)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:32.230291 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954c29d6-c677-4547-ac34-3214cb0abf9d admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:32.297826 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-954c29d6-c677-4547-ac34-3214cb0abf9d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db7d670d431242eaa6ed5049d454f18a', 'request_id': 'req-954c29d6-c677-4547-ac34-3214cb0abf9d'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:02:32.308729 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-954c29d6-c677-4547-ac34-3214cb0abf9d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db7d670d431242eaa6ed5049d454f18a', 'request_id': 'req-954c29d6-c677-4547-ac34-3214cb0abf9d'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:02:32.318576 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-954c29d6-c677-4547-ac34-3214cb0abf9d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db7d670d431242eaa6ed5049d454f18a', 'request_id': 'req-954c29d6-c677-4547-ac34-3214cb0abf9d'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:02:32.352404 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-954c29d6-c677-4547-ac34-3214cb0abf9d admin admin] The token cache is being invalidate because project db7d670d431242eaa6ed5049d454f18a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80154) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:02:32.352970 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-954c29d6-c677-4547-ac34-3214cb0abf9d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:02:32.355126 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-954c29d6-c677-4547-ac34-3214cb0abf9d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:02:32.363526 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1094/2190] 10.222.0.64 () {58 vars in 1233 bytes} [Tue Aug 2 10:02:32 2022] DELETE /identity/v3/projects/db7d670d431242eaa6ed5049d454f18a => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:02:32.365277 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108068ad-e35c-474d-ac9b-1439294e6163 service neutron] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:32.379177 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108068ad-e35c-474d-ac9b-1439294e6163 service neutron] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:32.469268 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10a5449-092f-4285-ab0d-891efc09a0f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:32.470185 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10a5449-092f-4285-ab0d-891efc09a0f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:32.470637 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10a5449-092f-4285-ab0d-891efc09a0f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:32.471742 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10a5449-092f-4285-ab0d-891efc09a0f6 admin admin] PATH_INFO: `/v3/users/3832be58608243298834b7d7a28aaa30` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:32.474452 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10a5449-092f-4285-ab0d-891efc09a0f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3832be58608243298834b7d7a28aaa30)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:32.476620 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10a5449-092f-4285-ab0d-891efc09a0f6 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:32.540592 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-f10a5449-092f-4285-ab0d-891efc09a0f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3832be58608243298834b7d7a28aaa30'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:02:32.551287 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-f10a5449-092f-4285-ab0d-891efc09a0f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3832be58608243298834b7d7a28aaa30'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:02:32.569100 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-f10a5449-092f-4285-ab0d-891efc09a0f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3832be58608243298834b7d7a28aaa30'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:02:32.570180 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-f10a5449-092f-4285-ab0d-891efc09a0f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3832be58608243298834b7d7a28aaa30'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:02:32.571114 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-f10a5449-092f-4285-ab0d-891efc09a0f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3832be58608243298834b7d7a28aaa30'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:02:32.593545 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:32.601466 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1097/2191] 10.222.0.64 () {58 vars in 1224 bytes} [Tue Aug 2 10:02:32 2022] DELETE /identity/v3/users/3832be58608243298834b7d7a28aaa30 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:02:33.102338 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954c29d6-c677-4547-ac34-3214cb0abf9d admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:33.104781 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-954c29d6-c677-4547-ac34-3214cb0abf9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:33.122971 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954c29d6-c677-4547-ac34-3214cb0abf9d admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:33.226381 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ea9e82-b6ca-4efa-af28-72b1ff7a2436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:33.227314 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ea9e82-b6ca-4efa-af28-72b1ff7a2436 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:33.227852 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ea9e82-b6ca-4efa-af28-72b1ff7a2436 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:33.228326 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ea9e82-b6ca-4efa-af28-72b1ff7a2436 admin admin] PATH_INFO: `/v3/projects/480f236591c14b3a9aaa322549bf1115` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:33.229931 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ea9e82-b6ca-4efa-af28-72b1ff7a2436 admin admin] RBAC: Authorizing `identity:delete_project(project_id=480f236591c14b3a9aaa322549bf1115)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:33.232837 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ea9e82-b6ca-4efa-af28-72b1ff7a2436 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:33.336979 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:33.344918 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-a7ea9e82-b6ca-4efa-af28-72b1ff7a2436 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '480f236591c14b3a9aaa322549bf1115', 'request_id': 'req-a7ea9e82-b6ca-4efa-af28-72b1ff7a2436'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:02:33.354933 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-a7ea9e82-b6ca-4efa-af28-72b1ff7a2436 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '480f236591c14b3a9aaa322549bf1115', 'request_id': 'req-a7ea9e82-b6ca-4efa-af28-72b1ff7a2436'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:02:33.363200 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-a7ea9e82-b6ca-4efa-af28-72b1ff7a2436 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '480f236591c14b3a9aaa322549bf1115', 'request_id': 'req-a7ea9e82-b6ca-4efa-af28-72b1ff7a2436'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:02:33.392492 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-a7ea9e82-b6ca-4efa-af28-72b1ff7a2436 admin admin] The token cache is being invalidate because project 480f236591c14b3a9aaa322549bf1115 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80154) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:02:33.393424 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-a7ea9e82-b6ca-4efa-af28-72b1ff7a2436 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:02:33.394188 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-a7ea9e82-b6ca-4efa-af28-72b1ff7a2436 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:02:33.395808 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1095/2192] 10.222.0.64 () {58 vars in 1233 bytes} [Tue Aug 2 10:02:33 2022] DELETE /identity/v3/projects/480f236591c14b3a9aaa322549bf1115 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:02:33.404423 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2575825f-1568-45a9-b15c-f9bd46577182 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:33.404974 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2575825f-1568-45a9-b15c-f9bd46577182 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:33.412494 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2575825f-1568-45a9-b15c-f9bd46577182 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:33.438464 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-2575825f-1568-45a9-b15c-f9bd46577182 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:02:33.439922 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-2575825f-1568-45a9-b15c-f9bd46577182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:33.551636 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1098/2193] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:02:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:02:33.559704 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96d3257-1370-4bdd-81ff-9ada50ffffc9 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:33.560152 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96d3257-1370-4bdd-81ff-9ada50ffffc9 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:33.561162 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96d3257-1370-4bdd-81ff-9ada50ffffc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:33.586481 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-c96d3257-1370-4bdd-81ff-9ada50ffffc9 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:02:33.587780 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-c96d3257-1370-4bdd-81ff-9ada50ffffc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:33.599192 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1096/2194] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:02:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:02:33.603524 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2575825f-1568-45a9-b15c-f9bd46577182 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:33.618410 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2575825f-1568-45a9-b15c-f9bd46577182 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:33.626793 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84ff00d-7d96-484e-8eb6-e4f2e58664fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:33.628359 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84ff00d-7d96-484e-8eb6-e4f2e58664fe admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:33.628769 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84ff00d-7d96-484e-8eb6-e4f2e58664fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:33.629054 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84ff00d-7d96-484e-8eb6-e4f2e58664fe admin admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:33.631513 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84ff00d-7d96-484e-8eb6-e4f2e58664fe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:33.631828 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84ff00d-7d96-484e-8eb6-e4f2e58664fe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:33.633554 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84ff00d-7d96-484e-8eb6-e4f2e58664fe admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:33.650527 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1099/2195] 10.222.0.64 () {58 vars in 1153 bytes} [Tue Aug 2 10:02:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:02:33.652488 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96d3257-1370-4bdd-81ff-9ada50ffffc9 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:33.663421 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96d3257-1370-4bdd-81ff-9ada50ffffc9 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:33.669687 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6735165e-6968-4a71-ae5a-09b7a6dcce14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:33.670309 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6735165e-6968-4a71-ae5a-09b7a6dcce14 admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:33.670789 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6735165e-6968-4a71-ae5a-09b7a6dcce14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:33.671201 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6735165e-6968-4a71-ae5a-09b7a6dcce14 admin admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:33.672038 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6735165e-6968-4a71-ae5a-09b7a6dcce14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:33.674213 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6735165e-6968-4a71-ae5a-09b7a6dcce14 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:33.712654 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1097/2196] 10.222.0.64 () {60 vars in 1153 bytes} [Tue Aug 2 10:02:33 2022] POST /identity/v3/projects => generated 407 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:02:33.715682 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84ff00d-7d96-484e-8eb6-e4f2e58664fe admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:33.727782 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84ff00d-7d96-484e-8eb6-e4f2e58664fe admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:33.795930 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e18b04-903c-45f2-ae35-f7699c06e991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:33.796685 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e18b04-903c-45f2-ae35-f7699c06e991 admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:33.796977 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e18b04-903c-45f2-ae35-f7699c06e991 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:33.797259 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e18b04-903c-45f2-ae35-f7699c06e991 admin admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:33.797955 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e18b04-903c-45f2-ae35-f7699c06e991 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:33.800184 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e18b04-903c-45f2-ae35-f7699c06e991 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:33.828947 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1100/2197] 10.222.0.64 () {60 vars in 1144 bytes} [Tue Aug 2 10:02:33 2022] POST /identity/v3/users => generated 304 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:02:33.833972 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6735165e-6968-4a71-ae5a-09b7a6dcce14 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:33.852273 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6735165e-6968-4a71-ae5a-09b7a6dcce14 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:33.861213 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d43288-3483-41f9-b12a-f423a2b7c62d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:33.862465 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d43288-3483-41f9-b12a-f423a2b7c62d admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:33.862763 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d43288-3483-41f9-b12a-f423a2b7c62d admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:33.863194 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d43288-3483-41f9-b12a-f423a2b7c62d admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:33.863974 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d43288-3483-41f9-b12a-f423a2b7c62d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:33.865761 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d43288-3483-41f9-b12a-f423a2b7c62d admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:33.879843 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1098/2198] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:02:33 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:02:33.890208 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e18b04-903c-45f2-ae35-f7699c06e991 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:33.909935 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e18b04-903c-45f2-ae35-f7699c06e991 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:33.918900 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fcec21-3c23-4df8-bea8-93ace7f1d55f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:33.919661 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fcec21-3c23-4df8-bea8-93ace7f1d55f admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:33.920023 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fcec21-3c23-4df8-bea8-93ace7f1d55f admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:33.920421 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fcec21-3c23-4df8-bea8-93ace7f1d55f admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:33.921218 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fcec21-3c23-4df8-bea8-93ace7f1d55f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:33.922932 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fcec21-3c23-4df8-bea8-93ace7f1d55f admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:33.946694 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1101/2199] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:02:33 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:02:33.947997 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d43288-3483-41f9-b12a-f423a2b7c62d admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:33.964300 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d43288-3483-41f9-b12a-f423a2b7c62d admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:33.978045 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec403ad9-47cf-4d07-8af9-34645e7a5d6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:33.978663 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec403ad9-47cf-4d07-8af9-34645e7a5d6a admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:33.979003 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec403ad9-47cf-4d07-8af9-34645e7a5d6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:33.979219 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec403ad9-47cf-4d07-8af9-34645e7a5d6a admin admin] PATH_INFO: `/v3/projects/d71e4d3b7e4d41eb8b5c9536ad34e06d/users/0dbb28bad4404492bed0205f90ae159c/roles/66c1074780954f8e87ed11a5c4e8c36a` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:34.003835 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec403ad9-47cf-4d07-8af9-34645e7a5d6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d71e4d3b7e4d41eb8b5c9536ad34e06d, user_id=0dbb28bad4404492bed0205f90ae159c, role_id=66c1074780954f8e87ed11a5c4e8c36a)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:34.006911 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec403ad9-47cf-4d07-8af9-34645e7a5d6a admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:34.031075 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1099/2200] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 10:02:33 2022] PUT /identity/v3/projects/d71e4d3b7e4d41eb8b5c9536ad34e06d/users/0dbb28bad4404492bed0205f90ae159c/roles/66c1074780954f8e87ed11a5c4e8c36a => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:02:34.043200 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c5a7eb-265c-4cd4-864d-df805f58f0fd None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:34.043561 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c5a7eb-265c-4cd4-864d-df805f58f0fd None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:34.043817 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c5a7eb-265c-4cd4-864d-df805f58f0fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:34.083731 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-26c5a7eb-265c-4cd4-864d-df805f58f0fd None None] MFA Rules not processed for user `0dbb28bad4404492bed0205f90ae159c`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:02:34.085093 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-26c5a7eb-265c-4cd4-864d-df805f58f0fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:34.168721 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:34.202821 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1102/2201] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:02:34 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:02:34.207719 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec403ad9-47cf-4d07-8af9-34645e7a5d6a admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:34.297688 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec403ad9-47cf-4d07-8af9-34645e7a5d6a admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:34.303183 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105e2691-a1e9-4e63-a177-25965afc034d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:34.304048 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105e2691-a1e9-4e63-a177-25965afc034d admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:34.304181 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105e2691-a1e9-4e63-a177-25965afc034d admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:34.304423 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105e2691-a1e9-4e63-a177-25965afc034d admin admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:34.305410 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105e2691-a1e9-4e63-a177-25965afc034d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:34.307061 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105e2691-a1e9-4e63-a177-25965afc034d admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:34.352019 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1100/2202] 10.222.0.64 () {60 vars in 1153 bytes} [Tue Aug 2 10:02:34 2022] POST /identity/v3/projects => generated 407 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:02:34.356117 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c5a7eb-265c-4cd4-864d-df805f58f0fd None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:34.368297 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c5a7eb-265c-4cd4-864d-df805f58f0fd None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:34.444609 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ec4b12-2ef1-48e7-bdea-90d327325190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:34.445508 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ec4b12-2ef1-48e7-bdea-90d327325190 admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:34.445803 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ec4b12-2ef1-48e7-bdea-90d327325190 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:34.446150 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ec4b12-2ef1-48e7-bdea-90d327325190 admin admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:34.446981 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ec4b12-2ef1-48e7-bdea-90d327325190 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:34.449544 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ec4b12-2ef1-48e7-bdea-90d327325190 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:34.484111 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105e2691-a1e9-4e63-a177-25965afc034d admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:34.484578 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1103/2203] 10.222.0.64 () {60 vars in 1144 bytes} [Tue Aug 2 10:02:34 2022] POST /identity/v3/users => generated 310 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:02:34.496004 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105e2691-a1e9-4e63-a177-25965afc034d admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:34.501479 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd34c58-1d66-4627-8f9f-31d8ab97f0e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:34.502726 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd34c58-1d66-4627-8f9f-31d8ab97f0e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:34.502726 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd34c58-1d66-4627-8f9f-31d8ab97f0e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:34.502945 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd34c58-1d66-4627-8f9f-31d8ab97f0e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:34.503717 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd34c58-1d66-4627-8f9f-31d8ab97f0e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:34.505779 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd34c58-1d66-4627-8f9f-31d8ab97f0e6 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:34.518065 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1101/2204] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:02:34 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:02:34.521659 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ec4b12-2ef1-48e7-bdea-90d327325190 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:34.542231 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ec4b12-2ef1-48e7-bdea-90d327325190 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:34.551594 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92409697-65e5-4cff-9f86-b8e56ac894ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:34.552313 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92409697-65e5-4cff-9f86-b8e56ac894ee admin admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:34.552610 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92409697-65e5-4cff-9f86-b8e56ac894ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:34.552893 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92409697-65e5-4cff-9f86-b8e56ac894ee admin admin] PATH_INFO: `/v3/projects/c8332be5e39b4824b09943fcc999c774/users/21ed939bff2c46fd8a2048bd5a99bc23/roles/1000cf318fa14915b8091599eaead5de` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:34.591242 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92409697-65e5-4cff-9f86-b8e56ac894ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8332be5e39b4824b09943fcc999c774, user_id=21ed939bff2c46fd8a2048bd5a99bc23, role_id=1000cf318fa14915b8091599eaead5de)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:34.601533 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92409697-65e5-4cff-9f86-b8e56ac894ee admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:34.643688 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1104/2205] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 10:02:34 2022] PUT /identity/v3/projects/c8332be5e39b4824b09943fcc999c774/users/21ed939bff2c46fd8a2048bd5a99bc23/roles/1000cf318fa14915b8091599eaead5de => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:02:34.645463 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd34c58-1d66-4627-8f9f-31d8ab97f0e6 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:34.663925 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd34c58-1d66-4627-8f9f-31d8ab97f0e6 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:34.793525 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:34.852724 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c81c7f7-a0a4-477c-a1ff-9fb8676f8d93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:34.854353 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c81c7f7-a0a4-477c-a1ff-9fb8676f8d93 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:34.854724 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c81c7f7-a0a4-477c-a1ff-9fb8676f8d93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:34.855078 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c81c7f7-a0a4-477c-a1ff-9fb8676f8d93 admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:34.855890 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c81c7f7-a0a4-477c-a1ff-9fb8676f8d93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:34.857912 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c81c7f7-a0a4-477c-a1ff-9fb8676f8d93 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:34.874017 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1102/2206] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:02:34 2022] GET /identity/v3/roles => generated 1420 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:02:34.879557 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92409697-65e5-4cff-9f86-b8e56ac894ee admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:34.897006 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92409697-65e5-4cff-9f86-b8e56ac894ee admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:34.903041 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da9c5a6-bb54-483b-9fc5-6711147f7a3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:34.903731 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da9c5a6-bb54-483b-9fc5-6711147f7a3f admin admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:34.904009 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da9c5a6-bb54-483b-9fc5-6711147f7a3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:34.904366 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da9c5a6-bb54-483b-9fc5-6711147f7a3f admin admin] PATH_INFO: `/v3/projects/c8332be5e39b4824b09943fcc999c774/users/21ed939bff2c46fd8a2048bd5a99bc23/roles/1000cf318fa14915b8091599eaead5de` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:34.906808 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da9c5a6-bb54-483b-9fc5-6711147f7a3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8332be5e39b4824b09943fcc999c774, user_id=21ed939bff2c46fd8a2048bd5a99bc23, role_id=1000cf318fa14915b8091599eaead5de)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:34.910929 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da9c5a6-bb54-483b-9fc5-6711147f7a3f admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:34.927366 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1105/2207] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 10:02:34 2022] PUT /identity/v3/projects/c8332be5e39b4824b09943fcc999c774/users/21ed939bff2c46fd8a2048bd5a99bc23/roles/1000cf318fa14915b8091599eaead5de => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:02:34.938491 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88abdf15-c617-45c5-8de9-9cbd7fbcab2b None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:34.938882 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88abdf15-c617-45c5-8de9-9cbd7fbcab2b None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:34.939438 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88abdf15-c617-45c5-8de9-9cbd7fbcab2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:34.988799 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-88abdf15-c617-45c5-8de9-9cbd7fbcab2b None None] MFA Rules not processed for user `21ed939bff2c46fd8a2048bd5a99bc23`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:02:34.990280 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-88abdf15-c617-45c5-8de9-9cbd7fbcab2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:35.003036 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da9c5a6-bb54-483b-9fc5-6711147f7a3f admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:35.006601 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-1da9c5a6-bb54-483b-9fc5-6711147f7a3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:35.118273 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da9c5a6-bb54-483b-9fc5-6711147f7a3f admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:35.124517 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d139776-c24a-46e1-a5de-14f6b90a87ca service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b770f073c86b4d88837a8b1f796a8b1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:35.125355 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d139776-c24a-46e1-a5de-14f6b90a87ca service placement] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:35.125798 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d139776-c24a-46e1-a5de-14f6b90a87ca service placement] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:35.126225 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d139776-c24a-46e1-a5de-14f6b90a87ca service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:35.128447 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-6d139776-c24a-46e1-a5de-14f6b90a87ca service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:35.134977 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1103/2208] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:02:34 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:02:35.147563 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88abdf15-c617-45c5-8de9-9cbd7fbcab2b None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:35.151660 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-88abdf15-c617-45c5-8de9-9cbd7fbcab2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:35.152660 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d139776-c24a-46e1-a5de-14f6b90a87ca service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:35.154501 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d139776-c24a-46e1-a5de-14f6b90a87ca service placement] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:35.169707 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1106/2209] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:02:35 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:02:35.233915 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88abdf15-c617-45c5-8de9-9cbd7fbcab2b None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:35.241829 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e17b105-ea88-41f4-85e8-7780159dfc87 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d7de62b9ee4039bdc79627ac7fb499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:35.242634 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e17b105-ea88-41f4-85e8-7780159dfc87 service cinder] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:35.242933 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e17b105-ea88-41f4-85e8-7780159dfc87 service cinder] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:35.243303 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e17b105-ea88-41f4-85e8-7780159dfc87 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:35.256578 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e17b105-ea88-41f4-85e8-7780159dfc87 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:35.257951 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e17b105-ea88-41f4-85e8-7780159dfc87 service cinder] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:35.274280 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1104/2210] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:02:35 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:02:35.557152 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d139776-c24a-46e1-a5de-14f6b90a87ca service placement] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:35.567841 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d139776-c24a-46e1-a5de-14f6b90a87ca service placement] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:35.572712 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3fd9f2-1d50-45aa-98f0-67fe55735ce2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d7de62b9ee4039bdc79627ac7fb499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:35.574166 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3fd9f2-1d50-45aa-98f0-67fe55735ce2 service cinder] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:35.574456 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3fd9f2-1d50-45aa-98f0-67fe55735ce2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:35.574720 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3fd9f2-1d50-45aa-98f0-67fe55735ce2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:35.621715 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:35.660155 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3fd9f2-1d50-45aa-98f0-67fe55735ce2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:35.662154 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3fd9f2-1d50-45aa-98f0-67fe55735ce2 service cinder] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:35.673441 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1107/2211] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:02:35 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:02:36.509154 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21d1569-d22b-4cc0-afdd-424e776d9119 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:36.509684 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21d1569-d22b-4cc0-afdd-424e776d9119 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:36.509807 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21d1569-d22b-4cc0-afdd-424e776d9119 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:36.547289 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-e21d1569-d22b-4cc0-afdd-424e776d9119 None None] MFA Rules not processed for user `8026b988ac8d4342b0d3fd2ccfae915c`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:02:36.548722 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-e21d1569-d22b-4cc0-afdd-424e776d9119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:36.613916 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:36.647048 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1105/2212] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:02:36 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:02:36.661586 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3fd9f2-1d50-45aa-98f0-67fe55735ce2 service cinder] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:36.663713 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-9a3fd9f2-1d50-45aa-98f0-67fe55735ce2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:36.756658 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3fd9f2-1d50-45aa-98f0-67fe55735ce2 service cinder] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:36.764871 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b036713b-d29c-4f8b-ad30-e5d572a49b70 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04fb1dbe260a4721aaa0bd0493260d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:36.765918 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b036713b-d29c-4f8b-ad30-e5d572a49b70 service glance] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:36.767369 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b036713b-d29c-4f8b-ad30-e5d572a49b70 service glance] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:36.768364 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b036713b-d29c-4f8b-ad30-e5d572a49b70 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:36.770921 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-b036713b-d29c-4f8b-ad30-e5d572a49b70 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:36.797613 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e6e806-1f32-40ab-8eb4-43e6cd6fa8f1 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:36.797905 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e6e806-1f32-40ab-8eb4-43e6cd6fa8f1 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:36.798183 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e6e806-1f32-40ab-8eb4-43e6cd6fa8f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:36.826536 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-59e6e806-1f32-40ab-8eb4-43e6cd6fa8f1 None None] MFA Rules not processed for user `5338bc8c594e4bc9bc41019226bf8d87`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:02:36.828809 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-59e6e806-1f32-40ab-8eb4-43e6cd6fa8f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:36.908213 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b036713b-d29c-4f8b-ad30-e5d572a49b70 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:36.910369 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b036713b-d29c-4f8b-ad30-e5d572a49b70 service glance] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:36.932750 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1108/2213] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:02:36 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:02:37.029832 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1106/2214] 10.222.0.64 () {58 vars in 1009 bytes} [Tue Aug 2 10:02:36 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:02:37.045239 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b036713b-d29c-4f8b-ad30-e5d572a49b70 service glance] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:37.047291 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-b036713b-d29c-4f8b-ad30-e5d572a49b70 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:37.165240 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b036713b-d29c-4f8b-ad30-e5d572a49b70 service glance] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:37.172157 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9be1643-9f98-461c-a79f-899f0402f5cb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a371e3df3764a4cbb1b8368aa74bd65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:37.172833 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9be1643-9f98-461c-a79f-899f0402f5cb service swift] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:37.174517 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9be1643-9f98-461c-a79f-899f0402f5cb service swift] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:37.174931 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9be1643-9f98-461c-a79f-899f0402f5cb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:37.175693 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2127c5fc-da31-4ffe-ae94-e18cfed84579 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:37.175992 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2127c5fc-da31-4ffe-ae94-e18cfed84579 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:37.176298 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2127c5fc-da31-4ffe-ae94-e18cfed84579 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:37.192126 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9be1643-9f98-461c-a79f-899f0402f5cb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:37.194597 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9be1643-9f98-461c-a79f-899f0402f5cb service swift] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:37.206141 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-2127c5fc-da31-4ffe-ae94-e18cfed84579 None None] MFA Rules not processed for user `5338bc8c594e4bc9bc41019226bf8d87`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:02:37.207729 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-2127c5fc-da31-4ffe-ae94-e18cfed84579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:37.221910 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1109/2215] 10.222.0.64 () {60 vars in 1384 bytes} [Tue Aug 2 10:02:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 02 10:02:37.224605 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1107/2216] 10.222.0.64 () {58 vars in 1009 bytes} [Tue Aug 2 10:02:37 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:02:37.257415 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9be1643-9f98-461c-a79f-899f0402f5cb service swift] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:37.273253 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9be1643-9f98-461c-a79f-899f0402f5cb service swift] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:37.281448 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0d7332-b39c-4ca2-9762-4723cff0bba9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a371e3df3764a4cbb1b8368aa74bd65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:37.282313 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0d7332-b39c-4ca2-9762-4723cff0bba9 service swift] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:37.282678 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0d7332-b39c-4ca2-9762-4723cff0bba9 service swift] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:37.283025 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0d7332-b39c-4ca2-9762-4723cff0bba9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:37.300876 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0d7332-b39c-4ca2-9762-4723cff0bba9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:37.303368 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0d7332-b39c-4ca2-9762-4723cff0bba9 service swift] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:37.322463 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1110/2217] 10.222.0.64 () {60 vars in 1384 bytes} [Tue Aug 2 10:02:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 02 10:02:40.976216 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bed55d-3a27-45bf-9520-08604f908545 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:40.977197 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bed55d-3a27-45bf-9520-08604f908545 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:40.977631 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bed55d-3a27-45bf-9520-08604f908545 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:41.035305 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-64bed55d-3a27-45bf-9520-08604f908545 None None] MFA Rules not processed for user `5bbba5ae48cc4ec39467df736ad1bbad`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:02:41.036621 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-64bed55d-3a27-45bf-9520-08604f908545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:41.171006 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1108/2218] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:02:40 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:02:47.311531 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=98 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:48.274608 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0d7332-b39c-4ca2-9762-4723cff0bba9 service swift] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:48.402773 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0d7332-b39c-4ca2-9762-4723cff0bba9 service swift] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:48.409329 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e775e1d-4865-45f5-bb51-5381059fc837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:48.410177 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e775e1d-4865-45f5-bb51-5381059fc837 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:48.414699 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e775e1d-4865-45f5-bb51-5381059fc837 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:48.415249 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e775e1d-4865-45f5-bb51-5381059fc837 admin admin] PATH_INFO: `/v3/users/0dbb28bad4404492bed0205f90ae159c` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:48.417327 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e775e1d-4865-45f5-bb51-5381059fc837 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0dbb28bad4404492bed0205f90ae159c)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:48.419635 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e775e1d-4865-45f5-bb51-5381059fc837 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:48.495402 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-5e775e1d-4865-45f5-bb51-5381059fc837 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0dbb28bad4404492bed0205f90ae159c'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:02:48.504440 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-5e775e1d-4865-45f5-bb51-5381059fc837 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0dbb28bad4404492bed0205f90ae159c'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:02:48.518641 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-5e775e1d-4865-45f5-bb51-5381059fc837 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0dbb28bad4404492bed0205f90ae159c'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:02:48.519589 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-5e775e1d-4865-45f5-bb51-5381059fc837 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0dbb28bad4404492bed0205f90ae159c'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:02:48.520424 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-5e775e1d-4865-45f5-bb51-5381059fc837 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0dbb28bad4404492bed0205f90ae159c'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:02:48.543532 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1111/2219] 10.222.0.64 () {58 vars in 1224 bytes} [Tue Aug 2 10:02:48 2022] DELETE /identity/v3/users/0dbb28bad4404492bed0205f90ae159c => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:02:48.548258 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7681418b-2c1f-4b65-94a9-d0a420d4912e None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:48.549376 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7681418b-2c1f-4b65-94a9-d0a420d4912e None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:48.549376 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7681418b-2c1f-4b65-94a9-d0a420d4912e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:48.578167 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-7681418b-2c1f-4b65-94a9-d0a420d4912e None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:02:48.580749 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-7681418b-2c1f-4b65-94a9-d0a420d4912e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:48.665827 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1109/2220] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:02:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:02:48.674979 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e775e1d-4865-45f5-bb51-5381059fc837 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:48.677360 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-5e775e1d-4865-45f5-bb51-5381059fc837 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:48.836643 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e775e1d-4865-45f5-bb51-5381059fc837 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:48.842687 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f6304e-431c-4994-9852-c4f8a20ba553 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:48.843328 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f6304e-431c-4994-9852-c4f8a20ba553 service neutron] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:48.843613 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f6304e-431c-4994-9852-c4f8a20ba553 service neutron] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:48.844133 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f6304e-431c-4994-9852-c4f8a20ba553 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:48.854871 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f6304e-431c-4994-9852-c4f8a20ba553 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:48.856350 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f6304e-431c-4994-9852-c4f8a20ba553 service neutron] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:48.871728 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1112/2221] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:02:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:02:49.492874 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7681418b-2c1f-4b65-94a9-d0a420d4912e None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:49.498019 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-7681418b-2c1f-4b65-94a9-d0a420d4912e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:49.525973 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7681418b-2c1f-4b65-94a9-d0a420d4912e None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:49.582587 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50866777-ad1d-48ab-bc29-b913e984d911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:49.583364 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50866777-ad1d-48ab-bc29-b913e984d911 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:49.583640 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50866777-ad1d-48ab-bc29-b913e984d911 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:49.583919 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50866777-ad1d-48ab-bc29-b913e984d911 admin admin] PATH_INFO: `/v3/projects/d71e4d3b7e4d41eb8b5c9536ad34e06d` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:49.585298 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50866777-ad1d-48ab-bc29-b913e984d911 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d71e4d3b7e4d41eb8b5c9536ad34e06d)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:49.602670 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50866777-ad1d-48ab-bc29-b913e984d911 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:49.658220 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:49.686047 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-50866777-ad1d-48ab-bc29-b913e984d911 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd71e4d3b7e4d41eb8b5c9536ad34e06d', 'request_id': 'req-50866777-ad1d-48ab-bc29-b913e984d911'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:02:49.696902 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-50866777-ad1d-48ab-bc29-b913e984d911 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd71e4d3b7e4d41eb8b5c9536ad34e06d', 'request_id': 'req-50866777-ad1d-48ab-bc29-b913e984d911'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:02:49.704194 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-50866777-ad1d-48ab-bc29-b913e984d911 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd71e4d3b7e4d41eb8b5c9536ad34e06d', 'request_id': 'req-50866777-ad1d-48ab-bc29-b913e984d911'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:02:49.722687 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-50866777-ad1d-48ab-bc29-b913e984d911 admin admin] The token cache is being invalidate because project d71e4d3b7e4d41eb8b5c9536ad34e06d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80154) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:02:49.722991 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-50866777-ad1d-48ab-bc29-b913e984d911 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:02:49.723742 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-50866777-ad1d-48ab-bc29-b913e984d911 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:02:49.730013 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f6304e-431c-4994-9852-c4f8a20ba553 service neutron] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:49.735089 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1110/2222] 10.222.0.64 () {58 vars in 1233 bytes} [Tue Aug 2 10:02:49 2022] DELETE /identity/v3/projects/d71e4d3b7e4d41eb8b5c9536ad34e06d => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:02:49.747562 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f6304e-431c-4994-9852-c4f8a20ba553 service neutron] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:49.769962 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:49.825138 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a9fcc7-9290-4ff9-8308-172ed7283fc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:49.825795 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a9fcc7-9290-4ff9-8308-172ed7283fc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:49.826084 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a9fcc7-9290-4ff9-8308-172ed7283fc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:49.826365 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a9fcc7-9290-4ff9-8308-172ed7283fc3 admin admin] PATH_INFO: `/v3/users/21ed939bff2c46fd8a2048bd5a99bc23` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:49.827601 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a9fcc7-9290-4ff9-8308-172ed7283fc3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=21ed939bff2c46fd8a2048bd5a99bc23)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:49.830143 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a9fcc7-9290-4ff9-8308-172ed7283fc3 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:49.903699 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-d1a9fcc7-9290-4ff9-8308-172ed7283fc3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21ed939bff2c46fd8a2048bd5a99bc23'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:02:49.918770 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-d1a9fcc7-9290-4ff9-8308-172ed7283fc3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21ed939bff2c46fd8a2048bd5a99bc23'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:02:49.962026 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-d1a9fcc7-9290-4ff9-8308-172ed7283fc3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21ed939bff2c46fd8a2048bd5a99bc23'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:02:49.965932 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-d1a9fcc7-9290-4ff9-8308-172ed7283fc3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21ed939bff2c46fd8a2048bd5a99bc23'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:02:49.965932 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-d1a9fcc7-9290-4ff9-8308-172ed7283fc3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21ed939bff2c46fd8a2048bd5a99bc23'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:02:50.012235 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1113/2223] 10.222.0.64 () {58 vars in 1224 bytes} [Tue Aug 2 10:02:49 2022] DELETE /identity/v3/users/21ed939bff2c46fd8a2048bd5a99bc23 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:02:50.536317 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50866777-ad1d-48ab-bc29-b913e984d911 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:50.537905 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-50866777-ad1d-48ab-bc29-b913e984d911 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:50.574562 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50866777-ad1d-48ab-bc29-b913e984d911 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:50.679896 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8edbd94-0c2a-44e1-82a0-413ee138a253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:50.681395 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8edbd94-0c2a-44e1-82a0-413ee138a253 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:50.682055 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8edbd94-0c2a-44e1-82a0-413ee138a253 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:50.682536 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8edbd94-0c2a-44e1-82a0-413ee138a253 admin admin] PATH_INFO: `/v3/projects/c8332be5e39b4824b09943fcc999c774` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:50.683967 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8edbd94-0c2a-44e1-82a0-413ee138a253 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8332be5e39b4824b09943fcc999c774)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:50.686262 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8edbd94-0c2a-44e1-82a0-413ee138a253 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:50.740379 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-d8edbd94-0c2a-44e1-82a0-413ee138a253 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8332be5e39b4824b09943fcc999c774', 'request_id': 'req-d8edbd94-0c2a-44e1-82a0-413ee138a253'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:02:50.746359 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-d8edbd94-0c2a-44e1-82a0-413ee138a253 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8332be5e39b4824b09943fcc999c774', 'request_id': 'req-d8edbd94-0c2a-44e1-82a0-413ee138a253'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:02:50.756162 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-d8edbd94-0c2a-44e1-82a0-413ee138a253 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8332be5e39b4824b09943fcc999c774', 'request_id': 'req-d8edbd94-0c2a-44e1-82a0-413ee138a253'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:02:50.776280 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-d8edbd94-0c2a-44e1-82a0-413ee138a253 admin admin] The token cache is being invalidate because project c8332be5e39b4824b09943fcc999c774 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80154) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:02:50.776426 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-d8edbd94-0c2a-44e1-82a0-413ee138a253 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:02:50.778238 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-d8edbd94-0c2a-44e1-82a0-413ee138a253 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:02:50.780307 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1111/2224] 10.222.0.64 () {58 vars in 1233 bytes} [Tue Aug 2 10:02:50 2022] DELETE /identity/v3/projects/c8332be5e39b4824b09943fcc999c774 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:02:50.785425 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd4ea24-c367-48b8-9bfe-c856d81bf654 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:50.785956 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd4ea24-c367-48b8-9bfe-c856d81bf654 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:50.786152 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd4ea24-c367-48b8-9bfe-c856d81bf654 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:50.816178 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-0bd4ea24-c367-48b8-9bfe-c856d81bf654 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:02:50.817629 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-0bd4ea24-c367-48b8-9bfe-c856d81bf654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:50.932576 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7470adba-8895-442f-98a4-9525f87ce642 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:50.933182 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7470adba-8895-442f-98a4-9525f87ce642 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:50.933671 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7470adba-8895-442f-98a4-9525f87ce642 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:50.955321 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1114/2225] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:02:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:02:50.963812 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80e18f5-d4aa-4cd0-af30-d25168007b2b None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:50.964122 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80e18f5-d4aa-4cd0-af30-d25168007b2b None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:50.964417 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80e18f5-d4aa-4cd0-af30-d25168007b2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:50.972873 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-7470adba-8895-442f-98a4-9525f87ce642 None None] MFA Rules not processed for user `3f9506226c4846628d32aadb0f6b86be`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:02:50.974590 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-7470adba-8895-442f-98a4-9525f87ce642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:50.997630 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-d80e18f5-d4aa-4cd0-af30-d25168007b2b None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:02:51.001931 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-d80e18f5-d4aa-4cd0-af30-d25168007b2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:51.011950 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1115/2226] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:02:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:02:51.019057 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80e18f5-d4aa-4cd0-af30-d25168007b2b None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:51.034784 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80e18f5-d4aa-4cd0-af30-d25168007b2b None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:51.040393 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd39fe4-2d0b-4cbe-9e6c-4272d82f4a24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:51.041381 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd39fe4-2d0b-4cbe-9e6c-4272d82f4a24 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:51.041793 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd39fe4-2d0b-4cbe-9e6c-4272d82f4a24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:51.042172 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd39fe4-2d0b-4cbe-9e6c-4272d82f4a24 admin admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:51.043245 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd39fe4-2d0b-4cbe-9e6c-4272d82f4a24 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:02:51.044069 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd39fe4-2d0b-4cbe-9e6c-4272d82f4a24 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:51.045908 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd39fe4-2d0b-4cbe-9e6c-4272d82f4a24 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:51.046383 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:51.068319 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1116/2227] 10.222.0.64 () {58 vars in 1153 bytes} [Tue Aug 2 10:02:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:02:51.077648 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1112/2228] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:02:50 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:02:51.085602 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd39fe4-2d0b-4cbe-9e6c-4272d82f4a24 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:51.116467 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd39fe4-2d0b-4cbe-9e6c-4272d82f4a24 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:51.132544 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6e11f0-907d-4531-b831-a72aa6a6a23a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:51.133218 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6e11f0-907d-4531-b831-a72aa6a6a23a admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:51.133568 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6e11f0-907d-4531-b831-a72aa6a6a23a admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:51.136222 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6e11f0-907d-4531-b831-a72aa6a6a23a admin admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:51.139087 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6e11f0-907d-4531-b831-a72aa6a6a23a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:51.142894 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6e11f0-907d-4531-b831-a72aa6a6a23a admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:51.183196 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:51.201543 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1117/2229] 10.222.0.64 () {60 vars in 1153 bytes} [Tue Aug 2 10:02:51 2022] POST /identity/v3/projects => generated 371 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:02:51.203305 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7470adba-8895-442f-98a4-9525f87ce642 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:51.217458 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7470adba-8895-442f-98a4-9525f87ce642 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:51.302937 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219177d9-4a7f-49fc-aeae-d0c3ddbacd0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:51.303752 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219177d9-4a7f-49fc-aeae-d0c3ddbacd0d admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:51.304171 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219177d9-4a7f-49fc-aeae-d0c3ddbacd0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:51.304562 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219177d9-4a7f-49fc-aeae-d0c3ddbacd0d admin admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:51.305358 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219177d9-4a7f-49fc-aeae-d0c3ddbacd0d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:51.307404 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219177d9-4a7f-49fc-aeae-d0c3ddbacd0d admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:51.333553 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1113/2230] 10.222.0.64 () {60 vars in 1144 bytes} [Tue Aug 2 10:02:51 2022] POST /identity/v3/users => generated 286 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:02:51.338869 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6e11f0-907d-4531-b831-a72aa6a6a23a admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:51.352138 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6e11f0-907d-4531-b831-a72aa6a6a23a admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:51.358728 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5643af1-496e-45f1-812b-56f3d0a0e316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:51.359558 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5643af1-496e-45f1-812b-56f3d0a0e316 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:51.360002 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5643af1-496e-45f1-812b-56f3d0a0e316 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:51.360395 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5643af1-496e-45f1-812b-56f3d0a0e316 admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:51.361517 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5643af1-496e-45f1-812b-56f3d0a0e316 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:51.363439 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5643af1-496e-45f1-812b-56f3d0a0e316 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:51.382217 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1118/2231] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:02:51 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:02:51.383395 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219177d9-4a7f-49fc-aeae-d0c3ddbacd0d admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:51.395071 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219177d9-4a7f-49fc-aeae-d0c3ddbacd0d admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:51.401492 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be263ea8-5a59-48cf-bb0d-e51d66802d20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:51.402262 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be263ea8-5a59-48cf-bb0d-e51d66802d20 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:51.402655 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be263ea8-5a59-48cf-bb0d-e51d66802d20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:51.403158 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be263ea8-5a59-48cf-bb0d-e51d66802d20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:51.403979 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be263ea8-5a59-48cf-bb0d-e51d66802d20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:51.405732 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be263ea8-5a59-48cf-bb0d-e51d66802d20 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:51.419447 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1114/2232] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:02:51 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:02:51.423480 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5643af1-496e-45f1-812b-56f3d0a0e316 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:51.435976 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5643af1-496e-45f1-812b-56f3d0a0e316 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:51.442971 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb6eefa-b268-4f8f-8ba4-740b6c2f0a1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:51.443803 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb6eefa-b268-4f8f-8ba4-740b6c2f0a1c admin admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:51.444368 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb6eefa-b268-4f8f-8ba4-740b6c2f0a1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:51.444767 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb6eefa-b268-4f8f-8ba4-740b6c2f0a1c admin admin] PATH_INFO: `/v3/projects/a1b51a961f354c53bcf991bcf73bed5f/users/ecfa61404d5f463fba7c08466885ad09/roles/66c1074780954f8e87ed11a5c4e8c36a` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:51.467684 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb6eefa-b268-4f8f-8ba4-740b6c2f0a1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1b51a961f354c53bcf991bcf73bed5f, user_id=ecfa61404d5f463fba7c08466885ad09, role_id=66c1074780954f8e87ed11a5c4e8c36a)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:51.471619 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb6eefa-b268-4f8f-8ba4-740b6c2f0a1c admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:51.496887 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1119/2233] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 10:02:51 2022] PUT /identity/v3/projects/a1b51a961f354c53bcf991bcf73bed5f/users/ecfa61404d5f463fba7c08466885ad09/roles/66c1074780954f8e87ed11a5c4e8c36a => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:02:51.503036 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1682b37e-d363-4e36-883f-ffacd8cc4cae None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:51.503451 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1682b37e-d363-4e36-883f-ffacd8cc4cae None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:51.503864 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1682b37e-d363-4e36-883f-ffacd8cc4cae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:51.528867 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-1682b37e-d363-4e36-883f-ffacd8cc4cae None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:02:51.530249 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-1682b37e-d363-4e36-883f-ffacd8cc4cae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:51.603946 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1115/2234] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:02:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:02:51.615440 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb6eefa-b268-4f8f-8ba4-740b6c2f0a1c admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:51.617763 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-efb6eefa-b268-4f8f-8ba4-740b6c2f0a1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:51.687739 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb6eefa-b268-4f8f-8ba4-740b6c2f0a1c admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:51.692221 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19956dac-ee20-46a3-9d3f-10cb782b0065 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:51.692872 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19956dac-ee20-46a3-9d3f-10cb782b0065 service neutron] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:51.693164 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19956dac-ee20-46a3-9d3f-10cb782b0065 service neutron] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:51.694196 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19956dac-ee20-46a3-9d3f-10cb782b0065 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:51.711964 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19956dac-ee20-46a3-9d3f-10cb782b0065 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:51.713590 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19956dac-ee20-46a3-9d3f-10cb782b0065 service neutron] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:51.725033 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1120/2235] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:02:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:02:52.792427 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c1d0ff-6539-47df-9a58-952173707b44 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:52.793376 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c1d0ff-6539-47df-9a58-952173707b44 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:52.794162 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c1d0ff-6539-47df-9a58-952173707b44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:52.841963 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-e8c1d0ff-6539-47df-9a58-952173707b44 None None] MFA Rules not processed for user `ecfa61404d5f463fba7c08466885ad09`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:02:52.843519 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-e8c1d0ff-6539-47df-9a58-952173707b44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:52.993799 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:53.009981 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1116/2236] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:02:52 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:02:53.013357 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19956dac-ee20-46a3-9d3f-10cb782b0065 service neutron] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:53.033374 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19956dac-ee20-46a3-9d3f-10cb782b0065 service neutron] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:53.042100 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c483bed-3707-4327-97d6-279a5d45d613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:53.042742 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c483bed-3707-4327-97d6-279a5d45d613 admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:53.043028 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c483bed-3707-4327-97d6-279a5d45d613 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:53.043304 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c483bed-3707-4327-97d6-279a5d45d613 admin admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:53.043980 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c483bed-3707-4327-97d6-279a5d45d613 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:53.045929 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c483bed-3707-4327-97d6-279a5d45d613 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:53.109005 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1121/2237] 10.222.0.64 () {60 vars in 1153 bytes} [Tue Aug 2 10:02:53 2022] POST /identity/v3/projects => generated 371 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:02:53.113396 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c1d0ff-6539-47df-9a58-952173707b44 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:53.128087 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c1d0ff-6539-47df-9a58-952173707b44 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:53.214816 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547893c0-4840-4d77-9711-1a6554b013aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:53.215446 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547893c0-4840-4d77-9711-1a6554b013aa admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:53.215941 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547893c0-4840-4d77-9711-1a6554b013aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:53.216289 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547893c0-4840-4d77-9711-1a6554b013aa admin admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:53.217273 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547893c0-4840-4d77-9711-1a6554b013aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:53.219095 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547893c0-4840-4d77-9711-1a6554b013aa admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:53.241811 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1117/2238] 10.222.0.64 () {60 vars in 1144 bytes} [Tue Aug 2 10:02:53 2022] POST /identity/v3/users => generated 292 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:02:53.245634 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c483bed-3707-4327-97d6-279a5d45d613 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:53.260090 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c483bed-3707-4327-97d6-279a5d45d613 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:53.265980 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455f9c8c-47fe-40ca-a0e5-257a98a7a56b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:53.270106 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455f9c8c-47fe-40ca-a0e5-257a98a7a56b admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:53.270406 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455f9c8c-47fe-40ca-a0e5-257a98a7a56b admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:53.270699 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455f9c8c-47fe-40ca-a0e5-257a98a7a56b admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:53.271424 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455f9c8c-47fe-40ca-a0e5-257a98a7a56b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:53.273296 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455f9c8c-47fe-40ca-a0e5-257a98a7a56b admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:53.286425 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1122/2239] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:02:53 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:02:53.290335 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547893c0-4840-4d77-9711-1a6554b013aa admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:53.300934 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547893c0-4840-4d77-9711-1a6554b013aa admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:53.309987 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd7a73b-48d0-43d9-9d49-9c5a235ab8b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:53.310841 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd7a73b-48d0-43d9-9d49-9c5a235ab8b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:53.311195 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd7a73b-48d0-43d9-9d49-9c5a235ab8b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:53.311542 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd7a73b-48d0-43d9-9d49-9c5a235ab8b8 admin admin] PATH_INFO: `/v3/projects/2d520335e75242da80da96ab0ea9e7ff/users/67e61c63fd7f4866a311981edb9f617e/roles/1000cf318fa14915b8091599eaead5de` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:53.330329 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd7a73b-48d0-43d9-9d49-9c5a235ab8b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d520335e75242da80da96ab0ea9e7ff, user_id=67e61c63fd7f4866a311981edb9f617e, role_id=1000cf318fa14915b8091599eaead5de)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:53.335345 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd7a73b-48d0-43d9-9d49-9c5a235ab8b8 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:53.358092 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1118/2240] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 10:02:53 2022] PUT /identity/v3/projects/2d520335e75242da80da96ab0ea9e7ff/users/67e61c63fd7f4866a311981edb9f617e/roles/1000cf318fa14915b8091599eaead5de => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:02:53.362015 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455f9c8c-47fe-40ca-a0e5-257a98a7a56b admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:53.373654 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455f9c8c-47fe-40ca-a0e5-257a98a7a56b admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:53.447798 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edee3955-9db5-40ea-8fb5-b6b686c55a5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:53.448841 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edee3955-9db5-40ea-8fb5-b6b686c55a5a admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:53.449132 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edee3955-9db5-40ea-8fb5-b6b686c55a5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:53.449479 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edee3955-9db5-40ea-8fb5-b6b686c55a5a admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:53.450327 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edee3955-9db5-40ea-8fb5-b6b686c55a5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:53.452262 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edee3955-9db5-40ea-8fb5-b6b686c55a5a admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:53.472241 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1123/2241] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:02:53 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:02:53.473741 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd7a73b-48d0-43d9-9d49-9c5a235ab8b8 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:53.486940 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd7a73b-48d0-43d9-9d49-9c5a235ab8b8 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:53.492847 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d596aca4-23ea-4136-9f94-429139bc49be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:53.493473 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d596aca4-23ea-4136-9f94-429139bc49be admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:53.493765 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d596aca4-23ea-4136-9f94-429139bc49be admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:53.494035 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d596aca4-23ea-4136-9f94-429139bc49be admin admin] PATH_INFO: `/v3/projects/2d520335e75242da80da96ab0ea9e7ff/users/67e61c63fd7f4866a311981edb9f617e/roles/1000cf318fa14915b8091599eaead5de` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:53.497706 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d596aca4-23ea-4136-9f94-429139bc49be admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d520335e75242da80da96ab0ea9e7ff, user_id=67e61c63fd7f4866a311981edb9f617e, role_id=1000cf318fa14915b8091599eaead5de)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:53.502254 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d596aca4-23ea-4136-9f94-429139bc49be admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:53.514716 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1119/2242] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 10:02:53 2022] PUT /identity/v3/projects/2d520335e75242da80da96ab0ea9e7ff/users/67e61c63fd7f4866a311981edb9f617e/roles/1000cf318fa14915b8091599eaead5de => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:02:54.571903 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3033f40b-d0f7-4e21-a8ec-29129f59aca9 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:54.573280 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3033f40b-d0f7-4e21-a8ec-29129f59aca9 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:54.574039 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3033f40b-d0f7-4e21-a8ec-29129f59aca9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:54.641479 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-3033f40b-d0f7-4e21-a8ec-29129f59aca9 None None] MFA Rules not processed for user `67e61c63fd7f4866a311981edb9f617e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:02:54.646279 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-3033f40b-d0f7-4e21-a8ec-29129f59aca9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:54.685048 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:54.821332 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ed2362-deed-48d4-bbf0-caf59cce5042 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:54.821887 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ed2362-deed-48d4-bbf0-caf59cce5042 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:54.821887 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ed2362-deed-48d4-bbf0-caf59cce5042 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:54.832019 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1124/2243] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:02:54 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:02:54.850604 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3033f40b-d0f7-4e21-a8ec-29129f59aca9 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:54.865357 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-3033f40b-d0f7-4e21-a8ec-29129f59aca9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:54.866691 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-a7ed2362-deed-48d4-bbf0-caf59cce5042 None None] MFA Rules not processed for user `5338bc8c594e4bc9bc41019226bf8d87`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:02:54.867855 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-a7ed2362-deed-48d4-bbf0-caf59cce5042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:55.004479 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1120/2244] 10.222.0.64 () {58 vars in 1009 bytes} [Tue Aug 2 10:02:54 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:02:55.036934 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ed2362-deed-48d4-bbf0-caf59cce5042 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:55.038189 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-a7ed2362-deed-48d4-bbf0-caf59cce5042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:02:55.056233 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3033f40b-d0f7-4e21-a8ec-29129f59aca9 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:55.066704 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b2de47-149c-48e9-9fd0-ff7ab2615dc2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d7de62b9ee4039bdc79627ac7fb499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:55.067720 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b2de47-149c-48e9-9fd0-ff7ab2615dc2 service cinder] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:55.069187 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b2de47-149c-48e9-9fd0-ff7ab2615dc2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:55.072605 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b2de47-149c-48e9-9fd0-ff7ab2615dc2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:55.140439 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:02:55.170806 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ed2362-deed-48d4-bbf0-caf59cce5042 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:55.184984 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdfeb4c-b649-4158-b83a-c56b528b5199 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a371e3df3764a4cbb1b8368aa74bd65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:55.185693 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdfeb4c-b649-4158-b83a-c56b528b5199 service swift] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:55.185978 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdfeb4c-b649-4158-b83a-c56b528b5199 service swift] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:55.186472 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdfeb4c-b649-4158-b83a-c56b528b5199 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:55.208807 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdfeb4c-b649-4158-b83a-c56b528b5199 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:55.210940 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdfeb4c-b649-4158-b83a-c56b528b5199 service swift] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:55.223785 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b2de47-149c-48e9-9fd0-ff7ab2615dc2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:55.225947 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b2de47-149c-48e9-9fd0-ff7ab2615dc2 service cinder] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:55.238029 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1121/2245] 10.222.0.64 () {60 vars in 1384 bytes} [Tue Aug 2 10:02:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 02 10:02:55.251328 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1125/2246] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:02:54 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:02:58.318283 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdfeb4c-b649-4158-b83a-c56b528b5199 service swift] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:02:58.330194 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdfeb4c-b649-4158-b83a-c56b528b5199 service swift] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:02:58.340634 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a95c8e-f885-4373-bbaf-b3f4a496b77e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d7de62b9ee4039bdc79627ac7fb499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:02:58.341272 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a95c8e-f885-4373-bbaf-b3f4a496b77e service cinder] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:02:58.341566 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a95c8e-f885-4373-bbaf-b3f4a496b77e service cinder] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:02:58.341844 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a95c8e-f885-4373-bbaf-b3f4a496b77e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:02:58.359722 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a95c8e-f885-4373-bbaf-b3f4a496b77e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:02:58.362353 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a95c8e-f885-4373-bbaf-b3f4a496b77e service cinder] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:02:58.375918 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1122/2247] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:02:58 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:03:01.004149 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb741d8e-a19e-4a8d-b843-a9a0af1a00d8 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:01.004723 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb741d8e-a19e-4a8d-b843-a9a0af1a00d8 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:01.004944 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb741d8e-a19e-4a8d-b843-a9a0af1a00d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:01.026636 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-cb741d8e-a19e-4a8d-b843-a9a0af1a00d8 None None] MFA Rules not processed for user `5338bc8c594e4bc9bc41019226bf8d87`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:03:01.027853 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-cb741d8e-a19e-4a8d-b843-a9a0af1a00d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:03:01.035045 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1126/2248] 10.222.0.64 () {58 vars in 967 bytes} [Tue Aug 2 10:03:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:03:01.047052 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a95c8e-f885-4373-bbaf-b3f4a496b77e service cinder] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:01.059438 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a95c8e-f885-4373-bbaf-b3f4a496b77e service cinder] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:01.065366 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b4c972-6cdb-4c31-8d03-f986d4f33dac service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a371e3df3764a4cbb1b8368aa74bd65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:01.065994 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b4c972-6cdb-4c31-8d03-f986d4f33dac service swift] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:01.066278 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b4c972-6cdb-4c31-8d03-f986d4f33dac service swift] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:01.066559 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b4c972-6cdb-4c31-8d03-f986d4f33dac service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:01.075726 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b4c972-6cdb-4c31-8d03-f986d4f33dac service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:01.077565 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b4c972-6cdb-4c31-8d03-f986d4f33dac service swift] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:01.087423 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1123/2249] 10.222.0.64 () {60 vars in 1384 bytes} [Tue Aug 2 10:03:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 02 10:03:03.255559 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8cd791-c13e-4d37-b4c0-4b40f079fd23 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:03.256059 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8cd791-c13e-4d37-b4c0-4b40f079fd23 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:03.256501 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8cd791-c13e-4d37-b4c0-4b40f079fd23 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:03.258675 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1127/2250] 10.222.0.64 () {54 vars in 921 bytes} [Tue Aug 2 10:03:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:03:03.299241 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b003630b-48a7-4dc9-b38c-df53dfdffac6 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:03.299801 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b003630b-48a7-4dc9-b38c-df53dfdffac6 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:03.299907 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b003630b-48a7-4dc9-b38c-df53dfdffac6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:03.320714 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-b003630b-48a7-4dc9-b38c-df53dfdffac6 None None] MFA Rules not processed for user `b770f073c86b4d88837a8b1f796a8b1d`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:03:03.322106 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-b003630b-48a7-4dc9-b38c-df53dfdffac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:03:03.338558 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8cd791-c13e-4d37-b4c0-4b40f079fd23 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:03.341048 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-6d8cd791-c13e-4d37-b4c0-4b40f079fd23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:03:03.413638 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1124/2251] 10.222.0.64 () {58 vars in 1018 bytes} [Tue Aug 2 10:03:03 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:03:03.425000 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8cd791-c13e-4d37-b4c0-4b40f079fd23 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:03.426130 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b003630b-48a7-4dc9-b38c-df53dfdffac6 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:03.427719 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-b003630b-48a7-4dc9-b38c-df53dfdffac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:03:03.429671 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b865cd6-b530-49dd-b8c2-97b00ad963fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:03.432672 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b865cd6-b530-49dd-b8c2-97b00ad963fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:03.432672 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b865cd6-b530-49dd-b8c2-97b00ad963fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:03.432672 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b865cd6-b530-49dd-b8c2-97b00ad963fd admin admin] PATH_INFO: `/v3/users/ede2486b45f44ac3b0bdab2d40695bee` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:03.432672 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b865cd6-b530-49dd-b8c2-97b00ad963fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=ede2486b45f44ac3b0bdab2d40695bee)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:03.434048 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b865cd6-b530-49dd-b8c2-97b00ad963fd admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:03.439720 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:03.460214 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b003630b-48a7-4dc9-b38c-df53dfdffac6 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:03.469215 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55011d3-a489-457b-932d-b4d3263fb2b6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b770f073c86b4d88837a8b1f796a8b1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:03.470950 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55011d3-a489-457b-932d-b4d3263fb2b6 service placement] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:03.471570 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55011d3-a489-457b-932d-b4d3263fb2b6 service placement] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:03.471849 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55011d3-a489-457b-932d-b4d3263fb2b6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:03.482790 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55011d3-a489-457b-932d-b4d3263fb2b6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:03.485059 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55011d3-a489-457b-932d-b4d3263fb2b6 service placement] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:03.497431 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-7b865cd6-b530-49dd-b8c2-97b00ad963fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ede2486b45f44ac3b0bdab2d40695bee'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:03:03.497969 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1125/2252] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:03:03 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:03:03.505787 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-7b865cd6-b530-49dd-b8c2-97b00ad963fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ede2486b45f44ac3b0bdab2d40695bee'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:03:03.515274 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-7b865cd6-b530-49dd-b8c2-97b00ad963fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ede2486b45f44ac3b0bdab2d40695bee'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:03:03.516055 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-7b865cd6-b530-49dd-b8c2-97b00ad963fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ede2486b45f44ac3b0bdab2d40695bee'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:03:03.517276 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-7b865cd6-b530-49dd-b8c2-97b00ad963fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ede2486b45f44ac3b0bdab2d40695bee'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:03:03.550900 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1128/2253] 10.222.0.64 () {58 vars in 1224 bytes} [Tue Aug 2 10:03:03 2022] DELETE /identity/v3/users/ede2486b45f44ac3b0bdab2d40695bee => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:03:03.831824 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55011d3-a489-457b-932d-b4d3263fb2b6 service placement] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:03.864480 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55011d3-a489-457b-932d-b4d3263fb2b6 service placement] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:03.943798 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9653a919-0f70-402a-a552-6ef731b6a34c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:03.944676 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9653a919-0f70-402a-a552-6ef731b6a34c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:03.945139 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9653a919-0f70-402a-a552-6ef731b6a34c admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:03.945714 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9653a919-0f70-402a-a552-6ef731b6a34c admin admin] PATH_INFO: `/v3/projects/1cac2783154e4570b1fbfba61b13ed92` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:03.948442 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9653a919-0f70-402a-a552-6ef731b6a34c admin admin] RBAC: Authorizing `identity:delete_project(project_id=1cac2783154e4570b1fbfba61b13ed92)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:03.950615 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9653a919-0f70-402a-a552-6ef731b6a34c admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:04.008680 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:04.010984 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-9653a919-0f70-402a-a552-6ef731b6a34c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1cac2783154e4570b1fbfba61b13ed92', 'request_id': 'req-9653a919-0f70-402a-a552-6ef731b6a34c'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:03:04.021483 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-9653a919-0f70-402a-a552-6ef731b6a34c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1cac2783154e4570b1fbfba61b13ed92', 'request_id': 'req-9653a919-0f70-402a-a552-6ef731b6a34c'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:03:04.029448 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-9653a919-0f70-402a-a552-6ef731b6a34c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1cac2783154e4570b1fbfba61b13ed92', 'request_id': 'req-9653a919-0f70-402a-a552-6ef731b6a34c'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:03:04.044235 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-9653a919-0f70-402a-a552-6ef731b6a34c admin admin] The token cache is being invalidate because project 1cac2783154e4570b1fbfba61b13ed92 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80154) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:03:04.044576 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-9653a919-0f70-402a-a552-6ef731b6a34c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:03:04.045427 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-9653a919-0f70-402a-a552-6ef731b6a34c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:03:04.047164 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1126/2254] 10.222.0.64 () {58 vars in 1233 bytes} [Tue Aug 2 10:03:03 2022] DELETE /identity/v3/projects/1cac2783154e4570b1fbfba61b13ed92 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:03:04.052491 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe9d1dd-7ca1-469f-be55-e114de060ca0 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:04.052792 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe9d1dd-7ca1-469f-be55-e114de060ca0 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:04.053080 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe9d1dd-7ca1-469f-be55-e114de060ca0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:04.079445 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-2fe9d1dd-7ca1-469f-be55-e114de060ca0 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:03:04.080741 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-2fe9d1dd-7ca1-469f-be55-e114de060ca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:03:04.185458 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1129/2255] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:03:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:03:04.196148 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487f46b4-c5b1-498c-9b28-f35c3746a9c4 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:04.196481 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487f46b4-c5b1-498c-9b28-f35c3746a9c4 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:04.196767 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487f46b4-c5b1-498c-9b28-f35c3746a9c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:04.220375 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-487f46b4-c5b1-498c-9b28-f35c3746a9c4 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:03:04.221604 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-487f46b4-c5b1-498c-9b28-f35c3746a9c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:03:04.242047 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1127/2256] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:03:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:03:04.246058 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe9d1dd-7ca1-469f-be55-e114de060ca0 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:04.257360 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe9d1dd-7ca1-469f-be55-e114de060ca0 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:04.262127 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ef6add-9333-4f85-8a6b-192b9a71bd04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:04.262751 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ef6add-9333-4f85-8a6b-192b9a71bd04 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:04.263067 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ef6add-9333-4f85-8a6b-192b9a71bd04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:04.263445 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ef6add-9333-4f85-8a6b-192b9a71bd04 admin admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:04.264147 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ef6add-9333-4f85-8a6b-192b9a71bd04 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:03:04.264491 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ef6add-9333-4f85-8a6b-192b9a71bd04 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:04.266113 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ef6add-9333-4f85-8a6b-192b9a71bd04 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:04.278405 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1130/2257] 10.222.0.64 () {58 vars in 1153 bytes} [Tue Aug 2 10:03:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:03:04.283624 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487f46b4-c5b1-498c-9b28-f35c3746a9c4 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:04.294815 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487f46b4-c5b1-498c-9b28-f35c3746a9c4 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:04.300996 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71c06b8-5e59-4c71-8c4e-67cad51c53e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:04.301639 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71c06b8-5e59-4c71-8c4e-67cad51c53e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:04.301899 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71c06b8-5e59-4c71-8c4e-67cad51c53e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:04.302173 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71c06b8-5e59-4c71-8c4e-67cad51c53e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:04.302841 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71c06b8-5e59-4c71-8c4e-67cad51c53e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:04.305620 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71c06b8-5e59-4c71-8c4e-67cad51c53e9 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:04.338119 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1128/2258] 10.222.0.64 () {60 vars in 1153 bytes} [Tue Aug 2 10:03:04 2022] POST /identity/v3/projects => generated 367 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:03:04.342142 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ef6add-9333-4f85-8a6b-192b9a71bd04 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:04.352918 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ef6add-9333-4f85-8a6b-192b9a71bd04 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:04.427354 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a47f993-62de-4bb6-ae8e-957327c5d4df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:04.428008 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a47f993-62de-4bb6-ae8e-957327c5d4df admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:04.428379 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a47f993-62de-4bb6-ae8e-957327c5d4df admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:04.428784 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a47f993-62de-4bb6-ae8e-957327c5d4df admin admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:04.429807 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a47f993-62de-4bb6-ae8e-957327c5d4df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:04.431749 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a47f993-62de-4bb6-ae8e-957327c5d4df admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:04.459940 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:04.468701 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1131/2259] 10.222.0.64 () {60 vars in 1144 bytes} [Tue Aug 2 10:03:04 2022] POST /identity/v3/users => generated 284 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:03:04.472190 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71c06b8-5e59-4c71-8c4e-67cad51c53e9 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:04.486022 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71c06b8-5e59-4c71-8c4e-67cad51c53e9 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:04.490655 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52614c78-87bd-422a-8087-3652e3fb4f0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:04.491274 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52614c78-87bd-422a-8087-3652e3fb4f0c admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:04.491583 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52614c78-87bd-422a-8087-3652e3fb4f0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:04.491830 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52614c78-87bd-422a-8087-3652e3fb4f0c admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:04.492564 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52614c78-87bd-422a-8087-3652e3fb4f0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:04.494185 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52614c78-87bd-422a-8087-3652e3fb4f0c admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:04.506261 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1129/2260] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:03:04 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:03:04.510162 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a47f993-62de-4bb6-ae8e-957327c5d4df admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:04.521861 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a47f993-62de-4bb6-ae8e-957327c5d4df admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:04.527132 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d06fd23-66d7-4ae6-8bf9-c469c4a4dbef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:04.527757 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d06fd23-66d7-4ae6-8bf9-c469c4a4dbef admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:04.528039 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d06fd23-66d7-4ae6-8bf9-c469c4a4dbef admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:04.528344 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d06fd23-66d7-4ae6-8bf9-c469c4a4dbef admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:04.529050 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d06fd23-66d7-4ae6-8bf9-c469c4a4dbef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:04.530652 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d06fd23-66d7-4ae6-8bf9-c469c4a4dbef admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:04.543015 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1132/2261] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:03:04 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:03:04.547031 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52614c78-87bd-422a-8087-3652e3fb4f0c admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:04.557972 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52614c78-87bd-422a-8087-3652e3fb4f0c admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:04.562643 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913fffa6-a0c4-4e59-a1e4-057e57cc75c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:04.563360 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913fffa6-a0c4-4e59-a1e4-057e57cc75c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:04.563634 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913fffa6-a0c4-4e59-a1e4-057e57cc75c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:04.563916 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913fffa6-a0c4-4e59-a1e4-057e57cc75c8 admin admin] PATH_INFO: `/v3/projects/31bef77a130f41059016cffa22985bec/users/139dec02ac59445ca6d28df4adafceed/roles/66c1074780954f8e87ed11a5c4e8c36a` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:04.580607 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913fffa6-a0c4-4e59-a1e4-057e57cc75c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31bef77a130f41059016cffa22985bec, user_id=139dec02ac59445ca6d28df4adafceed, role_id=66c1074780954f8e87ed11a5c4e8c36a)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:04.583260 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913fffa6-a0c4-4e59-a1e4-057e57cc75c8 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:04.597606 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1130/2262] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 10:03:04 2022] PUT /identity/v3/projects/31bef77a130f41059016cffa22985bec/users/139dec02ac59445ca6d28df4adafceed/roles/66c1074780954f8e87ed11a5c4e8c36a => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:03:04.607784 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7761153f-c7f9-4c6c-a9f0-da49602d4545 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:04.608078 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7761153f-c7f9-4c6c-a9f0-da49602d4545 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:04.608449 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7761153f-c7f9-4c6c-a9f0-da49602d4545 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:04.637931 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-7761153f-c7f9-4c6c-a9f0-da49602d4545 None None] MFA Rules not processed for user `139dec02ac59445ca6d28df4adafceed`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:03:04.639177 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-7761153f-c7f9-4c6c-a9f0-da49602d4545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:03:04.705697 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1133/2263] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:03:04 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:03:04.717074 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913fffa6-a0c4-4e59-a1e4-057e57cc75c8 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:04.718767 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-913fffa6-a0c4-4e59-a1e4-057e57cc75c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:03:04.779227 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913fffa6-a0c4-4e59-a1e4-057e57cc75c8 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:04.783857 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eec1f7b-5c95-4f17-858b-c1c478629f7c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04fb1dbe260a4721aaa0bd0493260d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:04.784504 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eec1f7b-5c95-4f17-858b-c1c478629f7c service glance] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:04.784790 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eec1f7b-5c95-4f17-858b-c1c478629f7c service glance] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:04.785069 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eec1f7b-5c95-4f17-858b-c1c478629f7c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:04.793335 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eec1f7b-5c95-4f17-858b-c1c478629f7c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:04.794764 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eec1f7b-5c95-4f17-858b-c1c478629f7c service glance] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:04.803629 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1131/2264] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:03:04 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:03:14.702227 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=37 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:14.799711 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:DELETE=4,keystone:SELECT=60,keystone:INSERT=3 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:15.287468 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7761153f-c7f9-4c6c-a9f0-da49602d4545 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:15.289124 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-7761153f-c7f9-4c6c-a9f0-da49602d4545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:03:15.384608 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7761153f-c7f9-4c6c-a9f0-da49602d4545 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:15.390537 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a826e8a8-17bc-4630-8442-46186510e8c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:15.391212 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a826e8a8-17bc-4630-8442-46186510e8c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:15.391495 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a826e8a8-17bc-4630-8442-46186510e8c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:15.391765 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a826e8a8-17bc-4630-8442-46186510e8c9 admin admin] PATH_INFO: `/v3/users/ecfa61404d5f463fba7c08466885ad09` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:15.392961 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a826e8a8-17bc-4630-8442-46186510e8c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ecfa61404d5f463fba7c08466885ad09)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:15.394878 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a826e8a8-17bc-4630-8442-46186510e8c9 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:15.451325 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-a826e8a8-17bc-4630-8442-46186510e8c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ecfa61404d5f463fba7c08466885ad09'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:03:15.459565 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-a826e8a8-17bc-4630-8442-46186510e8c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ecfa61404d5f463fba7c08466885ad09'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:03:15.469871 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-a826e8a8-17bc-4630-8442-46186510e8c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ecfa61404d5f463fba7c08466885ad09'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:03:15.470592 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-a826e8a8-17bc-4630-8442-46186510e8c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ecfa61404d5f463fba7c08466885ad09'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:03:15.471210 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-a826e8a8-17bc-4630-8442-46186510e8c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ecfa61404d5f463fba7c08466885ad09'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:03:15.494733 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1134/2265] 10.222.0.64 () {58 vars in 1224 bytes} [Tue Aug 2 10:03:15 2022] DELETE /identity/v3/users/ecfa61404d5f463fba7c08466885ad09 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:03:15.810754 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eec1f7b-5c95-4f17-858b-c1c478629f7c service glance] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:15.822443 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eec1f7b-5c95-4f17-858b-c1c478629f7c service glance] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:15.908797 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71157b4-ebfb-4183-bee7-4b56f7dc1cf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:15.909448 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71157b4-ebfb-4183-bee7-4b56f7dc1cf9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:15.909733 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71157b4-ebfb-4183-bee7-4b56f7dc1cf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:15.910012 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71157b4-ebfb-4183-bee7-4b56f7dc1cf9 admin admin] PATH_INFO: `/v3/projects/a1b51a961f354c53bcf991bcf73bed5f` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:15.911179 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71157b4-ebfb-4183-bee7-4b56f7dc1cf9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1b51a961f354c53bcf991bcf73bed5f)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:15.913196 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71157b4-ebfb-4183-bee7-4b56f7dc1cf9 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:15.958624 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-f71157b4-ebfb-4183-bee7-4b56f7dc1cf9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1b51a961f354c53bcf991bcf73bed5f', 'request_id': 'req-f71157b4-ebfb-4183-bee7-4b56f7dc1cf9'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:03:15.963899 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-f71157b4-ebfb-4183-bee7-4b56f7dc1cf9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1b51a961f354c53bcf991bcf73bed5f', 'request_id': 'req-f71157b4-ebfb-4183-bee7-4b56f7dc1cf9'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:03:15.970661 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-f71157b4-ebfb-4183-bee7-4b56f7dc1cf9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1b51a961f354c53bcf991bcf73bed5f', 'request_id': 'req-f71157b4-ebfb-4183-bee7-4b56f7dc1cf9'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:03:15.984502 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-f71157b4-ebfb-4183-bee7-4b56f7dc1cf9 admin admin] The token cache is being invalidate because project a1b51a961f354c53bcf991bcf73bed5f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80154) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:03:15.984790 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-f71157b4-ebfb-4183-bee7-4b56f7dc1cf9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:03:15.985424 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-f71157b4-ebfb-4183-bee7-4b56f7dc1cf9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:03:15.986989 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1132/2266] 10.222.0.64 () {58 vars in 1233 bytes} [Tue Aug 2 10:03:15 2022] DELETE /identity/v3/projects/a1b51a961f354c53bcf991bcf73bed5f => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:03:15.991128 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a826e8a8-17bc-4630-8442-46186510e8c9 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:15.992582 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-a826e8a8-17bc-4630-8442-46186510e8c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:03:16.005280 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a826e8a8-17bc-4630-8442-46186510e8c9 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:16.064430 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d01084-1be6-476c-b394-e4d0c625e8d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:16.065079 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d01084-1be6-476c-b394-e4d0c625e8d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:16.065363 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d01084-1be6-476c-b394-e4d0c625e8d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:16.065637 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d01084-1be6-476c-b394-e4d0c625e8d4 admin admin] PATH_INFO: `/v3/users/67e61c63fd7f4866a311981edb9f617e` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:16.067327 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d01084-1be6-476c-b394-e4d0c625e8d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=67e61c63fd7f4866a311981edb9f617e)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:16.068914 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d01084-1be6-476c-b394-e4d0c625e8d4 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:16.095107 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:16.124342 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-e7d01084-1be6-476c-b394-e4d0c625e8d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '67e61c63fd7f4866a311981edb9f617e'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:03:16.132043 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-e7d01084-1be6-476c-b394-e4d0c625e8d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '67e61c63fd7f4866a311981edb9f617e'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:03:16.139903 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-e7d01084-1be6-476c-b394-e4d0c625e8d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '67e61c63fd7f4866a311981edb9f617e'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:03:16.140633 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-e7d01084-1be6-476c-b394-e4d0c625e8d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '67e61c63fd7f4866a311981edb9f617e'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:03:16.141841 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-e7d01084-1be6-476c-b394-e4d0c625e8d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '67e61c63fd7f4866a311981edb9f617e'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:03:16.166625 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1135/2267] 10.222.0.64 () {58 vars in 1224 bytes} [Tue Aug 2 10:03:15 2022] DELETE /identity/v3/users/67e61c63fd7f4866a311981edb9f617e => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:03:16.351225 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71157b4-ebfb-4183-bee7-4b56f7dc1cf9 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:16.352709 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-f71157b4-ebfb-4183-bee7-4b56f7dc1cf9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:03:16.366580 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71157b4-ebfb-4183-bee7-4b56f7dc1cf9 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:16.431998 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dec007f-7df3-4433-9e6f-01a4cf8d7310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:16.432697 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dec007f-7df3-4433-9e6f-01a4cf8d7310 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:16.432987 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dec007f-7df3-4433-9e6f-01a4cf8d7310 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:16.433266 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dec007f-7df3-4433-9e6f-01a4cf8d7310 admin admin] PATH_INFO: `/v3/projects/2d520335e75242da80da96ab0ea9e7ff` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:16.434528 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dec007f-7df3-4433-9e6f-01a4cf8d7310 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d520335e75242da80da96ab0ea9e7ff)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:16.436536 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dec007f-7df3-4433-9e6f-01a4cf8d7310 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:16.475046 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:16.488840 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-0dec007f-7df3-4433-9e6f-01a4cf8d7310 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d520335e75242da80da96ab0ea9e7ff', 'request_id': 'req-0dec007f-7df3-4433-9e6f-01a4cf8d7310'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:03:16.494374 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-0dec007f-7df3-4433-9e6f-01a4cf8d7310 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d520335e75242da80da96ab0ea9e7ff', 'request_id': 'req-0dec007f-7df3-4433-9e6f-01a4cf8d7310'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:03:16.501144 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-0dec007f-7df3-4433-9e6f-01a4cf8d7310 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d520335e75242da80da96ab0ea9e7ff', 'request_id': 'req-0dec007f-7df3-4433-9e6f-01a4cf8d7310'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:03:16.513873 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-0dec007f-7df3-4433-9e6f-01a4cf8d7310 admin admin] The token cache is being invalidate because project 2d520335e75242da80da96ab0ea9e7ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80154) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:03:16.514204 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-0dec007f-7df3-4433-9e6f-01a4cf8d7310 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:03:16.514842 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-0dec007f-7df3-4433-9e6f-01a4cf8d7310 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:03:16.517074 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1133/2268] 10.222.0.64 () {58 vars in 1233 bytes} [Tue Aug 2 10:03:16 2022] DELETE /identity/v3/projects/2d520335e75242da80da96ab0ea9e7ff => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:03:16.521907 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad72b39-cd8b-4854-a237-28018c25be22 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:16.522229 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad72b39-cd8b-4854-a237-28018c25be22 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:16.522504 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad72b39-cd8b-4854-a237-28018c25be22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:16.542693 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-6ad72b39-cd8b-4854-a237-28018c25be22 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:03:16.543907 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-6ad72b39-cd8b-4854-a237-28018c25be22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:03:16.608586 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1136/2269] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:03:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:03:16.679212 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199a3cc3-8fd1-4b39-82c7-fd545cf8ee9a None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:16.679501 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199a3cc3-8fd1-4b39-82c7-fd545cf8ee9a None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:16.679776 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199a3cc3-8fd1-4b39-82c7-fd545cf8ee9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:16.700762 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-199a3cc3-8fd1-4b39-82c7-fd545cf8ee9a None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:03:16.702323 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-199a3cc3-8fd1-4b39-82c7-fd545cf8ee9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:03:16.711674 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1134/2270] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:03:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:03:16.715912 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad72b39-cd8b-4854-a237-28018c25be22 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:16.726113 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad72b39-cd8b-4854-a237-28018c25be22 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:16.730704 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e3ded2-a0f4-4cac-b480-05d271f87aef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:16.731315 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e3ded2-a0f4-4cac-b480-05d271f87aef admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:16.731599 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e3ded2-a0f4-4cac-b480-05d271f87aef admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:16.731873 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e3ded2-a0f4-4cac-b480-05d271f87aef admin admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:16.732606 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e3ded2-a0f4-4cac-b480-05d271f87aef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:03:16.732910 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e3ded2-a0f4-4cac-b480-05d271f87aef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:16.734505 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e3ded2-a0f4-4cac-b480-05d271f87aef admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:16.750162 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1137/2271] 10.222.0.64 () {58 vars in 1153 bytes} [Tue Aug 2 10:03:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:03:16.751872 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199a3cc3-8fd1-4b39-82c7-fd545cf8ee9a None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:16.768538 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199a3cc3-8fd1-4b39-82c7-fd545cf8ee9a None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:16.774532 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97635671-5587-458f-a38a-ffce8800d852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:16.775159 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97635671-5587-458f-a38a-ffce8800d852 admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:16.775441 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97635671-5587-458f-a38a-ffce8800d852 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:16.775713 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97635671-5587-458f-a38a-ffce8800d852 admin admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:16.777041 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97635671-5587-458f-a38a-ffce8800d852 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:16.778438 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97635671-5587-458f-a38a-ffce8800d852 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:16.812465 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1135/2272] 10.222.0.64 () {60 vars in 1153 bytes} [Tue Aug 2 10:03:16 2022] POST /identity/v3/projects => generated 385 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:03:16.814287 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e3ded2-a0f4-4cac-b480-05d271f87aef admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:16.830403 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e3ded2-a0f4-4cac-b480-05d271f87aef admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:16.923141 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d7a041-80ba-4840-8d48-f4d1d778376c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:16.924185 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d7a041-80ba-4840-8d48-f4d1d778376c admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:16.924542 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d7a041-80ba-4840-8d48-f4d1d778376c admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:16.925046 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d7a041-80ba-4840-8d48-f4d1d778376c admin admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:16.926233 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d7a041-80ba-4840-8d48-f4d1d778376c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:16.927841 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d7a041-80ba-4840-8d48-f4d1d778376c admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:16.953854 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1138/2273] 10.222.0.64 () {60 vars in 1144 bytes} [Tue Aug 2 10:03:16 2022] POST /identity/v3/users => generated 293 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:03:16.957788 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97635671-5587-458f-a38a-ffce8800d852 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:16.971261 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97635671-5587-458f-a38a-ffce8800d852 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:16.979916 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98515072-8590-4978-b2f7-5eeaf5df48e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:16.980591 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98515072-8590-4978-b2f7-5eeaf5df48e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:16.980947 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98515072-8590-4978-b2f7-5eeaf5df48e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:16.981388 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98515072-8590-4978-b2f7-5eeaf5df48e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:16.982180 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98515072-8590-4978-b2f7-5eeaf5df48e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:16.983788 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98515072-8590-4978-b2f7-5eeaf5df48e2 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:16.995807 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1136/2274] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:03:16 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:03:16.999845 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d7a041-80ba-4840-8d48-f4d1d778376c admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:17.013742 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d7a041-80ba-4840-8d48-f4d1d778376c admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:17.019116 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f5dcae-1336-4643-92b7-1b77663bd115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:17.019739 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f5dcae-1336-4643-92b7-1b77663bd115 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:17.020030 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f5dcae-1336-4643-92b7-1b77663bd115 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:17.020359 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f5dcae-1336-4643-92b7-1b77663bd115 admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:17.021225 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f5dcae-1336-4643-92b7-1b77663bd115 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:17.022739 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f5dcae-1336-4643-92b7-1b77663bd115 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:17.039252 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1139/2275] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:03:16 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:03:17.043631 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98515072-8590-4978-b2f7-5eeaf5df48e2 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:17.054881 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98515072-8590-4978-b2f7-5eeaf5df48e2 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:17.061731 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ecb39a-306b-4450-afe5-deabeb4bfd14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:17.062523 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ecb39a-306b-4450-afe5-deabeb4bfd14 admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:17.062957 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ecb39a-306b-4450-afe5-deabeb4bfd14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:17.063772 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ecb39a-306b-4450-afe5-deabeb4bfd14 admin admin] PATH_INFO: `/v3/projects/e7e8778a3b794e7da869cc01e7156741/users/aa82227b5a0842bd8cb4b71e446a89ed/roles/66c1074780954f8e87ed11a5c4e8c36a` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:17.080478 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ecb39a-306b-4450-afe5-deabeb4bfd14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7e8778a3b794e7da869cc01e7156741, user_id=aa82227b5a0842bd8cb4b71e446a89ed, role_id=66c1074780954f8e87ed11a5c4e8c36a)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:17.085361 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ecb39a-306b-4450-afe5-deabeb4bfd14 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:17.105180 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1137/2276] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 10:03:17 2022] PUT /identity/v3/projects/e7e8778a3b794e7da869cc01e7156741/users/aa82227b5a0842bd8cb4b71e446a89ed/roles/66c1074780954f8e87ed11a5c4e8c36a => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:03:17.116080 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f5ca2a-9707-4a16-894a-c594ec5cce7c None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:17.116394 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f5ca2a-9707-4a16-894a-c594ec5cce7c None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:17.116673 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f5ca2a-9707-4a16-894a-c594ec5cce7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:17.119498 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ecb39a-306b-4450-afe5-deabeb4bfd14 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:17.121254 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-19ecb39a-306b-4450-afe5-deabeb4bfd14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:03:17.126102 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:17.148492 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-d6f5ca2a-9707-4a16-894a-c594ec5cce7c None None] MFA Rules not processed for user `aa82227b5a0842bd8cb4b71e446a89ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:03:17.149712 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-d6f5ca2a-9707-4a16-894a-c594ec5cce7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:03:17.180483 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ecb39a-306b-4450-afe5-deabeb4bfd14 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:17.184146 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c29681-605d-4172-b787-2e4f4f86d932 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:17.185157 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c29681-605d-4172-b787-2e4f4f86d932 service neutron] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:17.185157 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c29681-605d-4172-b787-2e4f4f86d932 service neutron] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:17.185363 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c29681-605d-4172-b787-2e4f4f86d932 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:17.187076 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-37c29681-605d-4172-b787-2e4f4f86d932 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:03:17.221584 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1140/2277] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:03:17 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:03:17.232798 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f5ca2a-9707-4a16-894a-c594ec5cce7c None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:17.234355 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-d6f5ca2a-9707-4a16-894a-c594ec5cce7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:03:17.256800 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c29681-605d-4172-b787-2e4f4f86d932 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:17.258318 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c29681-605d-4172-b787-2e4f4f86d932 service neutron] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:17.268182 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1138/2278] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:03:17 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:03:17.294723 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f5ca2a-9707-4a16-894a-c594ec5cce7c None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:17.299128 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1642ff4c-93d4-4d0c-98eb-4b424c7b37ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d7de62b9ee4039bdc79627ac7fb499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:17.299739 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1642ff4c-93d4-4d0c-98eb-4b424c7b37ca service cinder] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:17.300031 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1642ff4c-93d4-4d0c-98eb-4b424c7b37ca service cinder] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:17.300343 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1642ff4c-93d4-4d0c-98eb-4b424c7b37ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:17.311616 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1642ff4c-93d4-4d0c-98eb-4b424c7b37ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:17.311616 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1642ff4c-93d4-4d0c-98eb-4b424c7b37ca service cinder] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:17.320193 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1141/2279] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:03:17 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:03:17.343563 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c29681-605d-4172-b787-2e4f4f86d932 service neutron] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:17.354109 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:17.417175 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c29681-605d-4172-b787-2e4f4f86d932 service neutron] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:17.422017 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd26ea0-57a1-44dc-9f77-a048037bfa47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:17.423224 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd26ea0-57a1-44dc-9f77-a048037bfa47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:17.423640 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd26ea0-57a1-44dc-9f77-a048037bfa47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:17.423921 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd26ea0-57a1-44dc-9f77-a048037bfa47 admin admin] PATH_INFO: `/v3/users/aa82227b5a0842bd8cb4b71e446a89ed` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:17.425419 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd26ea0-57a1-44dc-9f77-a048037bfa47 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa82227b5a0842bd8cb4b71e446a89ed)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:17.427514 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd26ea0-57a1-44dc-9f77-a048037bfa47 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:17.482927 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-5fd26ea0-57a1-44dc-9f77-a048037bfa47 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa82227b5a0842bd8cb4b71e446a89ed'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:03:17.489948 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-5fd26ea0-57a1-44dc-9f77-a048037bfa47 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa82227b5a0842bd8cb4b71e446a89ed'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:03:17.490632 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-5fd26ea0-57a1-44dc-9f77-a048037bfa47 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa82227b5a0842bd8cb4b71e446a89ed'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:03:17.499292 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-5fd26ea0-57a1-44dc-9f77-a048037bfa47 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa82227b5a0842bd8cb4b71e446a89ed'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:03:17.511694 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-5fd26ea0-57a1-44dc-9f77-a048037bfa47 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa82227b5a0842bd8cb4b71e446a89ed'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:03:17.518165 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1139/2280] 10.222.0.64 () {58 vars in 1224 bytes} [Tue Aug 2 10:03:17 2022] DELETE /identity/v3/users/aa82227b5a0842bd8cb4b71e446a89ed => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:03:17.523476 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cdbf10-915e-4ace-a656-384e6c459e55 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:17.523476 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cdbf10-915e-4ace-a656-384e6c459e55 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:17.523476 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cdbf10-915e-4ace-a656-384e6c459e55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:17.544650 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-57cdbf10-915e-4ace-a656-384e6c459e55 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:03:17.545870 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-57cdbf10-915e-4ace-a656-384e6c459e55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:03:17.606113 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1142/2281] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:03:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:03:17.618329 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd26ea0-57a1-44dc-9f77-a048037bfa47 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:17.620103 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-5fd26ea0-57a1-44dc-9f77-a048037bfa47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:03:17.686529 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd26ea0-57a1-44dc-9f77-a048037bfa47 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:17.690412 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372df0d9-bab5-4e44-9b3b-a838e0d9b9fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:17.691042 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372df0d9-bab5-4e44-9b3b-a838e0d9b9fb service neutron] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:17.691329 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372df0d9-bab5-4e44-9b3b-a838e0d9b9fb service neutron] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:17.691607 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372df0d9-bab5-4e44-9b3b-a838e0d9b9fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:17.701160 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372df0d9-bab5-4e44-9b3b-a838e0d9b9fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:17.702599 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372df0d9-bab5-4e44-9b3b-a838e0d9b9fb service neutron] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:17.712742 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1140/2282] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:03:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:03:18.021812 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cdbf10-915e-4ace-a656-384e6c459e55 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:18.023571 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-57cdbf10-915e-4ace-a656-384e6c459e55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:03:18.039361 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cdbf10-915e-4ace-a656-384e6c459e55 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:18.046371 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa6b49b-0f62-4718-a992-3602926c49df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:18.047931 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa6b49b-0f62-4718-a992-3602926c49df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:18.048154 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa6b49b-0f62-4718-a992-3602926c49df admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:18.048475 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa6b49b-0f62-4718-a992-3602926c49df admin admin] PATH_INFO: `/v3/projects/e7e8778a3b794e7da869cc01e7156741` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:18.049831 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa6b49b-0f62-4718-a992-3602926c49df admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7e8778a3b794e7da869cc01e7156741)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:18.052465 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa6b49b-0f62-4718-a992-3602926c49df admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:18.095928 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:18.114509 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-5fa6b49b-0f62-4718-a992-3602926c49df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7e8778a3b794e7da869cc01e7156741', 'request_id': 'req-5fa6b49b-0f62-4718-a992-3602926c49df'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:03:18.122151 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-5fa6b49b-0f62-4718-a992-3602926c49df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7e8778a3b794e7da869cc01e7156741', 'request_id': 'req-5fa6b49b-0f62-4718-a992-3602926c49df'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:03:18.129106 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-5fa6b49b-0f62-4718-a992-3602926c49df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7e8778a3b794e7da869cc01e7156741', 'request_id': 'req-5fa6b49b-0f62-4718-a992-3602926c49df'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:03:18.147342 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-5fa6b49b-0f62-4718-a992-3602926c49df admin admin] The token cache is being invalidate because project e7e8778a3b794e7da869cc01e7156741 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80153) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:03:18.147639 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-5fa6b49b-0f62-4718-a992-3602926c49df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:03:18.148306 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-5fa6b49b-0f62-4718-a992-3602926c49df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:03:18.153570 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1143/2283] 10.222.0.64 () {58 vars in 1233 bytes} [Tue Aug 2 10:03:18 2022] DELETE /identity/v3/projects/e7e8778a3b794e7da869cc01e7156741 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:03:18.155417 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ac4a2c-97a7-4aa7-b994-a7ca3bc25feb None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:18.155907 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ac4a2c-97a7-4aa7-b994-a7ca3bc25feb None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:18.156006 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ac4a2c-97a7-4aa7-b994-a7ca3bc25feb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:18.187139 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-53ac4a2c-97a7-4aa7-b994-a7ca3bc25feb None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:03:18.190639 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-53ac4a2c-97a7-4aa7-b994-a7ca3bc25feb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:03:18.215474 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:18.257204 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1141/2284] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:03:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:03:18.264713 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c9ae14-ea4d-4725-b4ac-ab999a203419 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:18.265079 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c9ae14-ea4d-4725-b4ac-ab999a203419 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:18.265427 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c9ae14-ea4d-4725-b4ac-ab999a203419 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:18.286461 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-80c9ae14-ea4d-4725-b4ac-ab999a203419 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:03:18.287773 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-80c9ae14-ea4d-4725-b4ac-ab999a203419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:03:18.296211 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1144/2285] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:03:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:03:18.301210 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ac4a2c-97a7-4aa7-b994-a7ca3bc25feb None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:18.311096 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ac4a2c-97a7-4aa7-b994-a7ca3bc25feb None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:18.316643 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fac3f4-8a96-4e05-9c95-b518afe2bc3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:18.317383 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fac3f4-8a96-4e05-9c95-b518afe2bc3a admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:18.317747 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fac3f4-8a96-4e05-9c95-b518afe2bc3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:18.318100 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fac3f4-8a96-4e05-9c95-b518afe2bc3a admin admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:18.318906 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fac3f4-8a96-4e05-9c95-b518afe2bc3a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:03:18.319288 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fac3f4-8a96-4e05-9c95-b518afe2bc3a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:18.321063 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fac3f4-8a96-4e05-9c95-b518afe2bc3a admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:18.334511 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1142/2286] 10.222.0.64 () {58 vars in 1153 bytes} [Tue Aug 2 10:03:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:03:18.338968 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c9ae14-ea4d-4725-b4ac-ab999a203419 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:18.349728 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c9ae14-ea4d-4725-b4ac-ab999a203419 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:18.354405 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a06717-ee86-448a-b8d1-97bc63c1d928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:18.355018 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a06717-ee86-448a-b8d1-97bc63c1d928 admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:18.355300 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a06717-ee86-448a-b8d1-97bc63c1d928 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:18.355723 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a06717-ee86-448a-b8d1-97bc63c1d928 admin admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:18.356455 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a06717-ee86-448a-b8d1-97bc63c1d928 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:18.358353 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a06717-ee86-448a-b8d1-97bc63c1d928 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:18.388232 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1145/2287] 10.222.0.64 () {60 vars in 1153 bytes} [Tue Aug 2 10:03:18 2022] POST /identity/v3/projects => generated 369 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:03:18.389605 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fac3f4-8a96-4e05-9c95-b518afe2bc3a admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:18.400064 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fac3f4-8a96-4e05-9c95-b518afe2bc3a admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:18.486384 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89967346-272a-41ee-a29d-a549c36d967c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:18.487035 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89967346-272a-41ee-a29d-a549c36d967c admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:18.487396 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89967346-272a-41ee-a29d-a549c36d967c admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:18.487679 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89967346-272a-41ee-a29d-a549c36d967c admin admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:18.488426 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89967346-272a-41ee-a29d-a549c36d967c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:18.490398 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89967346-272a-41ee-a29d-a549c36d967c admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:18.517021 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1143/2288] 10.222.0.64 () {60 vars in 1144 bytes} [Tue Aug 2 10:03:18 2022] POST /identity/v3/users => generated 285 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:03:18.521102 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a06717-ee86-448a-b8d1-97bc63c1d928 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:18.534550 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a06717-ee86-448a-b8d1-97bc63c1d928 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:18.539346 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e091d35-2b21-45c8-aca8-c60b02a5684a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:18.540028 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e091d35-2b21-45c8-aca8-c60b02a5684a admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:18.540348 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e091d35-2b21-45c8-aca8-c60b02a5684a admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:18.540627 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e091d35-2b21-45c8-aca8-c60b02a5684a admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:18.541462 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e091d35-2b21-45c8-aca8-c60b02a5684a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:18.543088 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e091d35-2b21-45c8-aca8-c60b02a5684a admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:18.554373 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1146/2289] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:03:18 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:03:18.558244 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89967346-272a-41ee-a29d-a549c36d967c admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:18.569100 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89967346-272a-41ee-a29d-a549c36d967c admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:18.581473 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9261186-a4b8-4cdc-8a11-4caceec675fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:18.581826 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9261186-a4b8-4cdc-8a11-4caceec675fa admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:18.582206 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9261186-a4b8-4cdc-8a11-4caceec675fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:18.582496 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9261186-a4b8-4cdc-8a11-4caceec675fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:18.583254 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9261186-a4b8-4cdc-8a11-4caceec675fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:18.584967 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9261186-a4b8-4cdc-8a11-4caceec675fa admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:18.597256 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1144/2290] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:03:18 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:03:18.601507 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e091d35-2b21-45c8-aca8-c60b02a5684a admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:18.612040 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e091d35-2b21-45c8-aca8-c60b02a5684a admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:18.616782 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e318ec-c9d1-4f2a-9ba3-109327a89007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:18.617475 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e318ec-c9d1-4f2a-9ba3-109327a89007 admin admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:18.617762 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e318ec-c9d1-4f2a-9ba3-109327a89007 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:18.618042 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e318ec-c9d1-4f2a-9ba3-109327a89007 admin admin] PATH_INFO: `/v3/projects/184910308dd24deaab38998d75f0d39f/users/edbd105491e742c7ae5505d41ac80abe/roles/66c1074780954f8e87ed11a5c4e8c36a` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:18.634292 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e318ec-c9d1-4f2a-9ba3-109327a89007 admin admin] RBAC: Authorizing `identity:create_grant(project_id=184910308dd24deaab38998d75f0d39f, user_id=edbd105491e742c7ae5505d41ac80abe, role_id=66c1074780954f8e87ed11a5c4e8c36a)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:18.637097 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e318ec-c9d1-4f2a-9ba3-109327a89007 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:18.652989 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1147/2291] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 10:03:18 2022] PUT /identity/v3/projects/184910308dd24deaab38998d75f0d39f/users/edbd105491e742c7ae5505d41ac80abe/roles/66c1074780954f8e87ed11a5c4e8c36a => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:03:18.664553 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e178d5-4113-4551-82f3-f2fbe1aa058c None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:18.664851 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e178d5-4113-4551-82f3-f2fbe1aa058c None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:18.665152 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e178d5-4113-4551-82f3-f2fbe1aa058c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:18.696864 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-04e178d5-4113-4551-82f3-f2fbe1aa058c None None] MFA Rules not processed for user `edbd105491e742c7ae5505d41ac80abe`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:03:18.698391 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-04e178d5-4113-4551-82f3-f2fbe1aa058c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:03:18.771896 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1145/2292] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:03:18 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:03:18.779695 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e318ec-c9d1-4f2a-9ba3-109327a89007 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:18.781440 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-d0e318ec-c9d1-4f2a-9ba3-109327a89007 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:03:18.842511 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e318ec-c9d1-4f2a-9ba3-109327a89007 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:18.847140 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421f525a-a7df-4c52-8f23-209acf36751e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04fb1dbe260a4721aaa0bd0493260d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:18.847883 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421f525a-a7df-4c52-8f23-209acf36751e service glance] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:18.848277 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421f525a-a7df-4c52-8f23-209acf36751e service glance] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:18.848661 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421f525a-a7df-4c52-8f23-209acf36751e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:18.857944 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421f525a-a7df-4c52-8f23-209acf36751e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:18.859398 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421f525a-a7df-4c52-8f23-209acf36751e service glance] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:18.869603 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1148/2293] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:03:18 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:03:28.764537 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=86,keystone:INSERT=3 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:28.865041 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=65,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:03:50.336911 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e178d5-4113-4551-82f3-f2fbe1aa058c None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:03:50.338304 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-04e178d5-4113-4551-82f3-f2fbe1aa058c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:03:50.394899 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e178d5-4113-4551-82f3-f2fbe1aa058c None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:03:50.398410 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82b4864-0801-4bd2-9894-ec6ba7253513 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:03:50.399086 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82b4864-0801-4bd2-9894-ec6ba7253513 service neutron] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:03:50.399363 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82b4864-0801-4bd2-9894-ec6ba7253513 service neutron] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:03:50.399641 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82b4864-0801-4bd2-9894-ec6ba7253513 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:03:50.401432 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-f82b4864-0801-4bd2-9894-ec6ba7253513 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:03:50.462679 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82b4864-0801-4bd2-9894-ec6ba7253513 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:03:50.464607 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82b4864-0801-4bd2-9894-ec6ba7253513 service neutron] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:03:50.475120 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1146/2294] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:03:50 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:04:00.470257 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=50 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:04:20.291787 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421f525a-a7df-4c52-8f23-209acf36751e service glance] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:20.355287 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421f525a-a7df-4c52-8f23-209acf36751e service glance] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:20.359808 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e149523e-d11d-4da8-b121-567f5fbee1fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:20.360493 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e149523e-d11d-4da8-b121-567f5fbee1fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:20.360782 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e149523e-d11d-4da8-b121-567f5fbee1fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:20.361550 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e149523e-d11d-4da8-b121-567f5fbee1fe admin admin] PATH_INFO: `/v3/users/edbd105491e742c7ae5505d41ac80abe` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:20.362713 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e149523e-d11d-4da8-b121-567f5fbee1fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=edbd105491e742c7ae5505d41ac80abe)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:20.364687 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e149523e-d11d-4da8-b121-567f5fbee1fe admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:20.411128 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-e149523e-d11d-4da8-b121-567f5fbee1fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'edbd105491e742c7ae5505d41ac80abe'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:04:20.417904 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-e149523e-d11d-4da8-b121-567f5fbee1fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'edbd105491e742c7ae5505d41ac80abe'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:04:20.425084 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-e149523e-d11d-4da8-b121-567f5fbee1fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'edbd105491e742c7ae5505d41ac80abe'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:04:20.425705 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-e149523e-d11d-4da8-b121-567f5fbee1fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'edbd105491e742c7ae5505d41ac80abe'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:04:20.426320 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-e149523e-d11d-4da8-b121-567f5fbee1fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'edbd105491e742c7ae5505d41ac80abe'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:04:20.443113 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1149/2295] 10.222.0.64 () {58 vars in 1224 bytes} [Tue Aug 2 10:04:20 2022] DELETE /identity/v3/users/edbd105491e742c7ae5505d41ac80abe => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:04:20.447942 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fe1a4d-865c-4b44-9dda-344283975ab4 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:20.448469 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fe1a4d-865c-4b44-9dda-344283975ab4 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:20.448570 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fe1a4d-865c-4b44-9dda-344283975ab4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:20.467798 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-28fe1a4d-865c-4b44-9dda-344283975ab4 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:04:20.469042 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-28fe1a4d-865c-4b44-9dda-344283975ab4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:04:20.503812 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e149523e-d11d-4da8-b121-567f5fbee1fe admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:20.505324 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-e149523e-d11d-4da8-b121-567f5fbee1fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:04:20.532567 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1147/2296] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:04:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:04:20.542910 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fe1a4d-865c-4b44-9dda-344283975ab4 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:20.544275 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-28fe1a4d-865c-4b44-9dda-344283975ab4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:04:20.572299 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e149523e-d11d-4da8-b121-567f5fbee1fe admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:20.577010 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c7583f-59b5-46e1-86cb-dd1e8726c3fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:20.577653 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c7583f-59b5-46e1-86cb-dd1e8726c3fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:20.577939 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c7583f-59b5-46e1-86cb-dd1e8726c3fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:20.578902 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c7583f-59b5-46e1-86cb-dd1e8726c3fc admin admin] PATH_INFO: `/v3/users/139dec02ac59445ca6d28df4adafceed` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:20.580130 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c7583f-59b5-46e1-86cb-dd1e8726c3fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=139dec02ac59445ca6d28df4adafceed)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:20.582097 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c7583f-59b5-46e1-86cb-dd1e8726c3fc admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:20.602541 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fe1a4d-865c-4b44-9dda-344283975ab4 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:20.603859 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:04:20.608029 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10501070-ed9f-4284-8973-c6bf81b174f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:20.609021 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10501070-ed9f-4284-8973-c6bf81b174f2 service neutron] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:20.609771 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10501070-ed9f-4284-8973-c6bf81b174f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:20.610468 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10501070-ed9f-4284-8973-c6bf81b174f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:20.622145 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10501070-ed9f-4284-8973-c6bf81b174f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:20.624306 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10501070-ed9f-4284-8973-c6bf81b174f2 service neutron] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:20.636896 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1148/2297] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:04:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:04:20.637292 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-48c7583f-59b5-46e1-86cb-dd1e8726c3fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '139dec02ac59445ca6d28df4adafceed'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:04:20.644788 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-48c7583f-59b5-46e1-86cb-dd1e8726c3fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '139dec02ac59445ca6d28df4adafceed'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:04:20.653934 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-48c7583f-59b5-46e1-86cb-dd1e8726c3fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '139dec02ac59445ca6d28df4adafceed'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:04:20.654711 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-48c7583f-59b5-46e1-86cb-dd1e8726c3fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '139dec02ac59445ca6d28df4adafceed'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:04:20.655185 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-48c7583f-59b5-46e1-86cb-dd1e8726c3fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '139dec02ac59445ca6d28df4adafceed'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:04:20.679921 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1150/2298] 10.222.0.64 () {58 vars in 1224 bytes} [Tue Aug 2 10:04:20 2022] DELETE /identity/v3/users/139dec02ac59445ca6d28df4adafceed => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:04:20.684542 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf051d50-72d4-4a85-8f96-35f92bc78ff9 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:20.684867 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf051d50-72d4-4a85-8f96-35f92bc78ff9 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:20.685173 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf051d50-72d4-4a85-8f96-35f92bc78ff9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:20.712329 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-cf051d50-72d4-4a85-8f96-35f92bc78ff9 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:04:20.713744 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-cf051d50-72d4-4a85-8f96-35f92bc78ff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:04:20.788400 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1149/2299] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:04:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:04:20.804942 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c7583f-59b5-46e1-86cb-dd1e8726c3fc admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:20.806406 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-48c7583f-59b5-46e1-86cb-dd1e8726c3fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:04:20.872807 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c7583f-59b5-46e1-86cb-dd1e8726c3fc admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:20.877603 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a719970-d674-46f4-808f-4bd473e66767 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:20.878235 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a719970-d674-46f4-808f-4bd473e66767 service neutron] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:20.878503 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a719970-d674-46f4-808f-4bd473e66767 service neutron] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:20.878783 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a719970-d674-46f4-808f-4bd473e66767 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:20.900067 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a719970-d674-46f4-808f-4bd473e66767 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:20.901700 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a719970-d674-46f4-808f-4bd473e66767 service neutron] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:20.913293 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1151/2300] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:04:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:04:20.930757 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf051d50-72d4-4a85-8f96-35f92bc78ff9 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:20.932333 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-cf051d50-72d4-4a85-8f96-35f92bc78ff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:04:20.946388 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf051d50-72d4-4a85-8f96-35f92bc78ff9 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:20.951370 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fa487e-6d9d-49e8-b472-5ae38e97e3bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:20.952055 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fa487e-6d9d-49e8-b472-5ae38e97e3bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:20.952389 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fa487e-6d9d-49e8-b472-5ae38e97e3bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:20.952675 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fa487e-6d9d-49e8-b472-5ae38e97e3bb admin admin] PATH_INFO: `/v3/projects/184910308dd24deaab38998d75f0d39f` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:20.954087 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fa487e-6d9d-49e8-b472-5ae38e97e3bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=184910308dd24deaab38998d75f0d39f)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:20.956569 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fa487e-6d9d-49e8-b472-5ae38e97e3bb admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:21.001750 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:04:21.010725 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-a7fa487e-6d9d-49e8-b472-5ae38e97e3bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '184910308dd24deaab38998d75f0d39f', 'request_id': 'req-a7fa487e-6d9d-49e8-b472-5ae38e97e3bb'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:04:21.016188 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-a7fa487e-6d9d-49e8-b472-5ae38e97e3bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '184910308dd24deaab38998d75f0d39f', 'request_id': 'req-a7fa487e-6d9d-49e8-b472-5ae38e97e3bb'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:04:21.023349 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-a7fa487e-6d9d-49e8-b472-5ae38e97e3bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '184910308dd24deaab38998d75f0d39f', 'request_id': 'req-a7fa487e-6d9d-49e8-b472-5ae38e97e3bb'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:04:21.037278 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-a7fa487e-6d9d-49e8-b472-5ae38e97e3bb admin admin] The token cache is being invalidate because project 184910308dd24deaab38998d75f0d39f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80154) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:04:21.037701 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-a7fa487e-6d9d-49e8-b472-5ae38e97e3bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:04:21.038358 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-a7fa487e-6d9d-49e8-b472-5ae38e97e3bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:04:21.040150 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1150/2301] 10.222.0.64 () {58 vars in 1233 bytes} [Tue Aug 2 10:04:20 2022] DELETE /identity/v3/projects/184910308dd24deaab38998d75f0d39f => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:04:21.044645 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634dd7b3-965f-4c52-b16d-0e1eaadc19f0 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:21.044956 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634dd7b3-965f-4c52-b16d-0e1eaadc19f0 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:21.045211 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634dd7b3-965f-4c52-b16d-0e1eaadc19f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:21.078814 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-634dd7b3-965f-4c52-b16d-0e1eaadc19f0 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:04:21.080792 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-634dd7b3-965f-4c52-b16d-0e1eaadc19f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:04:21.141455 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1152/2302] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:04:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:04:21.149188 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5953ef23-4e64-4353-9456-1d8d7fc4ceea None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:21.149495 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5953ef23-4e64-4353-9456-1d8d7fc4ceea None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:21.149769 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5953ef23-4e64-4353-9456-1d8d7fc4ceea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:21.169450 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-5953ef23-4e64-4353-9456-1d8d7fc4ceea None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:04:21.170658 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-5953ef23-4e64-4353-9456-1d8d7fc4ceea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:04:21.179048 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1151/2303] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:04:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:04:21.183225 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634dd7b3-965f-4c52-b16d-0e1eaadc19f0 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:21.194479 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634dd7b3-965f-4c52-b16d-0e1eaadc19f0 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:21.199817 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae06b6ff-3c1c-4949-a5ad-fedf1d56dd55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:21.200278 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5953ef23-4e64-4353-9456-1d8d7fc4ceea None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:21.200583 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae06b6ff-3c1c-4949-a5ad-fedf1d56dd55 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:21.200898 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae06b6ff-3c1c-4949-a5ad-fedf1d56dd55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:21.201176 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae06b6ff-3c1c-4949-a5ad-fedf1d56dd55 admin admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:21.201839 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-5953ef23-4e64-4353-9456-1d8d7fc4ceea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:04:21.201964 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae06b6ff-3c1c-4949-a5ad-fedf1d56dd55 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:04:21.202205 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae06b6ff-3c1c-4949-a5ad-fedf1d56dd55 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:21.203800 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae06b6ff-3c1c-4949-a5ad-fedf1d56dd55 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:21.215141 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5953ef23-4e64-4353-9456-1d8d7fc4ceea None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:21.218082 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1153/2304] 10.222.0.64 () {58 vars in 1153 bytes} [Tue Aug 2 10:04:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:04:21.222174 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae06b6ff-3c1c-4949-a5ad-fedf1d56dd55 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:21.224492 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c94b849-cf9c-46be-b5a0-7549405c2396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:21.225209 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c94b849-cf9c-46be-b5a0-7549405c2396 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:21.225562 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c94b849-cf9c-46be-b5a0-7549405c2396 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:21.225910 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c94b849-cf9c-46be-b5a0-7549405c2396 admin admin] PATH_INFO: `/v3/projects/31bef77a130f41059016cffa22985bec` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:21.227162 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c94b849-cf9c-46be-b5a0-7549405c2396 admin admin] RBAC: Authorizing `identity:delete_project(project_id=31bef77a130f41059016cffa22985bec)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:21.229912 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c94b849-cf9c-46be-b5a0-7549405c2396 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:21.232765 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae06b6ff-3c1c-4949-a5ad-fedf1d56dd55 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:21.239995 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855c4080-dddd-4d5b-a27c-cedd840dcb07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:21.240663 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855c4080-dddd-4d5b-a27c-cedd840dcb07 admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:21.240944 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855c4080-dddd-4d5b-a27c-cedd840dcb07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:21.241242 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855c4080-dddd-4d5b-a27c-cedd840dcb07 admin admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:21.241975 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855c4080-dddd-4d5b-a27c-cedd840dcb07 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:21.243778 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855c4080-dddd-4d5b-a27c-cedd840dcb07 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:21.272363 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1154/2305] 10.222.0.64 () {60 vars in 1153 bytes} [Tue Aug 2 10:04:21 2022] POST /identity/v3/projects => generated 375 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:04:21.276457 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855c4080-dddd-4d5b-a27c-cedd840dcb07 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:21.285418 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-3c94b849-cf9c-46be-b5a0-7549405c2396 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31bef77a130f41059016cffa22985bec', 'request_id': 'req-3c94b849-cf9c-46be-b5a0-7549405c2396'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:04:21.285883 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:DELETE=7,keystone:SELECT=91,keystone:INSERT=2 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:04:21.291575 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-3c94b849-cf9c-46be-b5a0-7549405c2396 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31bef77a130f41059016cffa22985bec', 'request_id': 'req-3c94b849-cf9c-46be-b5a0-7549405c2396'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:04:21.297898 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-3c94b849-cf9c-46be-b5a0-7549405c2396 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31bef77a130f41059016cffa22985bec', 'request_id': 'req-3c94b849-cf9c-46be-b5a0-7549405c2396'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:04:21.311002 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-3c94b849-cf9c-46be-b5a0-7549405c2396 admin admin] The token cache is being invalidate because project 31bef77a130f41059016cffa22985bec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80154) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:04:21.311299 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-3c94b849-cf9c-46be-b5a0-7549405c2396 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:04:21.311948 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-3c94b849-cf9c-46be-b5a0-7549405c2396 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:04:21.313566 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1152/2306] 10.222.0.64 () {58 vars in 1233 bytes} [Tue Aug 2 10:04:21 2022] DELETE /identity/v3/projects/31bef77a130f41059016cffa22985bec => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:04:21.352372 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855c4080-dddd-4d5b-a27c-cedd840dcb07 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:21.358969 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef0e99e-005b-4b39-80b5-c853cb4d91f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:21.359870 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef0e99e-005b-4b39-80b5-c853cb4d91f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:21.360227 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef0e99e-005b-4b39-80b5-c853cb4d91f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:21.360664 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef0e99e-005b-4b39-80b5-c853cb4d91f0 admin admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:21.362065 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef0e99e-005b-4b39-80b5-c853cb4d91f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:21.365040 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef0e99e-005b-4b39-80b5-c853cb4d91f0 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:21.391048 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1155/2307] 10.222.0.64 () {60 vars in 1144 bytes} [Tue Aug 2 10:04:21 2022] POST /identity/v3/users => generated 288 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:04:21.395276 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c94b849-cf9c-46be-b5a0-7549405c2396 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:21.396950 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-3c94b849-cf9c-46be-b5a0-7549405c2396 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:04:21.475862 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c94b849-cf9c-46be-b5a0-7549405c2396 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:21.481910 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb32f3ee-4cb5-417e-a0a5-a4c44fa1aa59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:21.482547 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb32f3ee-4cb5-417e-a0a5-a4c44fa1aa59 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:21.482887 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb32f3ee-4cb5-417e-a0a5-a4c44fa1aa59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:21.483167 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb32f3ee-4cb5-417e-a0a5-a4c44fa1aa59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:21.483884 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb32f3ee-4cb5-417e-a0a5-a4c44fa1aa59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:21.485565 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb32f3ee-4cb5-417e-a0a5-a4c44fa1aa59 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:21.502045 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1153/2308] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:04:21 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:04:21.506024 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef0e99e-005b-4b39-80b5-c853cb4d91f0 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:21.516843 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef0e99e-005b-4b39-80b5-c853cb4d91f0 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:21.521720 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78932468-375f-4041-b6b6-ad37fa5c5e22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:21.522427 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78932468-375f-4041-b6b6-ad37fa5c5e22 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:21.522766 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78932468-375f-4041-b6b6-ad37fa5c5e22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:21.523100 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78932468-375f-4041-b6b6-ad37fa5c5e22 admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:21.523889 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78932468-375f-4041-b6b6-ad37fa5c5e22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:21.525609 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78932468-375f-4041-b6b6-ad37fa5c5e22 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:21.536987 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1156/2309] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:04:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:04:21.541292 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb32f3ee-4cb5-417e-a0a5-a4c44fa1aa59 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:21.551823 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb32f3ee-4cb5-417e-a0a5-a4c44fa1aa59 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:21.556387 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8cac65-08b5-4ab6-9e75-69117f21fe72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:21.557040 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8cac65-08b5-4ab6-9e75-69117f21fe72 admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:21.557321 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8cac65-08b5-4ab6-9e75-69117f21fe72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:21.557600 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8cac65-08b5-4ab6-9e75-69117f21fe72 admin admin] PATH_INFO: `/v3/projects/25aeae85a0fe4a63950df2d312e8f53a/users/218dbcd89bc84c82a2dabc447668b0ba/roles/66c1074780954f8e87ed11a5c4e8c36a` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:21.572509 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8cac65-08b5-4ab6-9e75-69117f21fe72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25aeae85a0fe4a63950df2d312e8f53a, user_id=218dbcd89bc84c82a2dabc447668b0ba, role_id=66c1074780954f8e87ed11a5c4e8c36a)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:21.575252 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8cac65-08b5-4ab6-9e75-69117f21fe72 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:21.587783 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1154/2310] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 10:04:21 2022] PUT /identity/v3/projects/25aeae85a0fe4a63950df2d312e8f53a/users/218dbcd89bc84c82a2dabc447668b0ba/roles/66c1074780954f8e87ed11a5c4e8c36a => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:04:21.599888 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dca9a1-6526-4d0d-a880-ae1efbb13d0d None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:21.600069 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dca9a1-6526-4d0d-a880-ae1efbb13d0d None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:21.600401 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dca9a1-6526-4d0d-a880-ae1efbb13d0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:21.627982 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-21dca9a1-6526-4d0d-a880-ae1efbb13d0d None None] MFA Rules not processed for user `218dbcd89bc84c82a2dabc447668b0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:04:21.629214 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-21dca9a1-6526-4d0d-a880-ae1efbb13d0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:04:21.691517 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1157/2311] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:04:21 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:04:21.702528 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8cac65-08b5-4ab6-9e75-69117f21fe72 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:21.704356 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-df8cac65-08b5-4ab6-9e75-69117f21fe72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:04:21.723332 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:04:21.763717 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8cac65-08b5-4ab6-9e75-69117f21fe72 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:21.767421 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1da7fd-83c5-42b9-b5b7-ba8a9a4cc962 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d7de62b9ee4039bdc79627ac7fb499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:21.768026 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1da7fd-83c5-42b9-b5b7-ba8a9a4cc962 service cinder] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:21.768340 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1da7fd-83c5-42b9-b5b7-ba8a9a4cc962 service cinder] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:21.768617 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1da7fd-83c5-42b9-b5b7-ba8a9a4cc962 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:21.776663 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1da7fd-83c5-42b9-b5b7-ba8a9a4cc962 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:21.778083 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1da7fd-83c5-42b9-b5b7-ba8a9a4cc962 service cinder] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:21.787204 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1155/2312] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:04:21 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:04:26.244390 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc096579-9da5-4158-ba55-5e1ba0d7bc1e None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:26.244390 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc096579-9da5-4158-ba55-5e1ba0d7bc1e None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:26.244390 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc096579-9da5-4158-ba55-5e1ba0d7bc1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:26.267167 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-bc096579-9da5-4158-ba55-5e1ba0d7bc1e None None] MFA Rules not processed for user `dfe32e33711e42118f3ff5021762604f`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:04:26.268480 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-bc096579-9da5-4158-ba55-5e1ba0d7bc1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:04:26.331670 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1158/2313] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:04:26 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:04:30.442090 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1da7fd-83c5-42b9-b5b7-ba8a9a4cc962 service cinder] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:30.443642 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-2d1da7fd-83c5-42b9-b5b7-ba8a9a4cc962 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:04:30.501229 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1da7fd-83c5-42b9-b5b7-ba8a9a4cc962 service cinder] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:30.505631 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e925f4f-34f6-424d-9728-cc33b293c3f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:30.506259 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e925f4f-34f6-424d-9728-cc33b293c3f8 service neutron] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:30.506551 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e925f4f-34f6-424d-9728-cc33b293c3f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:30.506828 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e925f4f-34f6-424d-9728-cc33b293c3f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:30.508682 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-8e925f4f-34f6-424d-9728-cc33b293c3f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:04:30.586834 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e925f4f-34f6-424d-9728-cc33b293c3f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:30.588209 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e925f4f-34f6-424d-9728-cc33b293c3f8 service neutron] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:30.599339 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1156/2314] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:04:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:04:30.676584 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc096579-9da5-4158-ba55-5e1ba0d7bc1e None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:30.689511 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc096579-9da5-4158-ba55-5e1ba0d7bc1e None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:30.700586 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aacdc8a-82fb-4a89-bb3e-a76b14b863d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:30.701557 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aacdc8a-82fb-4a89-bb3e-a76b14b863d2 service neutron] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:30.702025 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aacdc8a-82fb-4a89-bb3e-a76b14b863d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:30.702461 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aacdc8a-82fb-4a89-bb3e-a76b14b863d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:30.705943 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-6aacdc8a-82fb-4a89-bb3e-a76b14b863d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:04:30.737924 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aacdc8a-82fb-4a89-bb3e-a76b14b863d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:30.742217 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aacdc8a-82fb-4a89-bb3e-a76b14b863d2 service neutron] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:30.760851 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1159/2315] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:04:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:04:31.824549 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e925f4f-34f6-424d-9728-cc33b293c3f8 service neutron] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:31.900899 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e925f4f-34f6-424d-9728-cc33b293c3f8 service neutron] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:31.905489 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1999081-9fe6-492f-b328-95d33f09d711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:31.906136 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1999081-9fe6-492f-b328-95d33f09d711 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:31.906426 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1999081-9fe6-492f-b328-95d33f09d711 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:31.906707 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1999081-9fe6-492f-b328-95d33f09d711 admin admin] PATH_INFO: `/v3/users/218dbcd89bc84c82a2dabc447668b0ba` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:31.907886 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1999081-9fe6-492f-b328-95d33f09d711 admin admin] RBAC: Authorizing `identity:delete_user(user_id=218dbcd89bc84c82a2dabc447668b0ba)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:31.909853 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1999081-9fe6-492f-b328-95d33f09d711 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:31.911471 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:04:31.961740 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-f1999081-9fe6-492f-b328-95d33f09d711 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '218dbcd89bc84c82a2dabc447668b0ba'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:04:31.968451 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-f1999081-9fe6-492f-b328-95d33f09d711 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '218dbcd89bc84c82a2dabc447668b0ba'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:04:31.969121 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-f1999081-9fe6-492f-b328-95d33f09d711 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '218dbcd89bc84c82a2dabc447668b0ba'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:04:31.977192 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-f1999081-9fe6-492f-b328-95d33f09d711 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '218dbcd89bc84c82a2dabc447668b0ba'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:04:31.990092 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-f1999081-9fe6-492f-b328-95d33f09d711 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '218dbcd89bc84c82a2dabc447668b0ba'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:04:31.996352 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1157/2316] 10.222.0.64 () {58 vars in 1224 bytes} [Tue Aug 2 10:04:31 2022] DELETE /identity/v3/users/218dbcd89bc84c82a2dabc447668b0ba => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:04:32.000824 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c183a21-c841-4903-b037-f345d2680215 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:32.001343 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c183a21-c841-4903-b037-f345d2680215 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:32.001447 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c183a21-c841-4903-b037-f345d2680215 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:32.014471 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:04:32.021899 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-0c183a21-c841-4903-b037-f345d2680215 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:04:32.023100 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-0c183a21-c841-4903-b037-f345d2680215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:04:32.085480 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1160/2317] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:04:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:04:32.093455 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1999081-9fe6-492f-b328-95d33f09d711 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:32.095741 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-f1999081-9fe6-492f-b328-95d33f09d711 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:04:32.154695 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1999081-9fe6-492f-b328-95d33f09d711 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:32.158858 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cb48c3-d47b-456b-90c1-f29f86e36a84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:32.159473 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cb48c3-d47b-456b-90c1-f29f86e36a84 service neutron] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:32.159754 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cb48c3-d47b-456b-90c1-f29f86e36a84 service neutron] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:32.160024 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cb48c3-d47b-456b-90c1-f29f86e36a84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:32.169351 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cb48c3-d47b-456b-90c1-f29f86e36a84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:32.170847 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cb48c3-d47b-456b-90c1-f29f86e36a84 service neutron] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:32.181182 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1158/2318] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:04:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:04:32.506563 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c183a21-c841-4903-b037-f345d2680215 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:32.508036 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-0c183a21-c841-4903-b037-f345d2680215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:04:32.521378 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c183a21-c841-4903-b037-f345d2680215 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:32.526307 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a20c024-cc8a-474b-81e3-19387de3e6a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:32.527158 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a20c024-cc8a-474b-81e3-19387de3e6a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:32.527455 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a20c024-cc8a-474b-81e3-19387de3e6a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:32.527739 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a20c024-cc8a-474b-81e3-19387de3e6a7 admin admin] PATH_INFO: `/v3/projects/25aeae85a0fe4a63950df2d312e8f53a` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:32.529231 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a20c024-cc8a-474b-81e3-19387de3e6a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=25aeae85a0fe4a63950df2d312e8f53a)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:32.531670 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a20c024-cc8a-474b-81e3-19387de3e6a7 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:32.582314 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-8a20c024-cc8a-474b-81e3-19387de3e6a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25aeae85a0fe4a63950df2d312e8f53a', 'request_id': 'req-8a20c024-cc8a-474b-81e3-19387de3e6a7'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:04:32.587957 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-8a20c024-cc8a-474b-81e3-19387de3e6a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25aeae85a0fe4a63950df2d312e8f53a', 'request_id': 'req-8a20c024-cc8a-474b-81e3-19387de3e6a7'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:04:32.594535 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-8a20c024-cc8a-474b-81e3-19387de3e6a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25aeae85a0fe4a63950df2d312e8f53a', 'request_id': 'req-8a20c024-cc8a-474b-81e3-19387de3e6a7'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:04:32.608402 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-8a20c024-cc8a-474b-81e3-19387de3e6a7 admin admin] The token cache is being invalidate because project 25aeae85a0fe4a63950df2d312e8f53a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80153) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:04:32.608693 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-8a20c024-cc8a-474b-81e3-19387de3e6a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:04:32.609348 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-8a20c024-cc8a-474b-81e3-19387de3e6a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:04:32.611003 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1161/2319] 10.222.0.64 () {58 vars in 1233 bytes} [Tue Aug 2 10:04:32 2022] DELETE /identity/v3/projects/25aeae85a0fe4a63950df2d312e8f53a => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:04:39.156573 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88db9be4-0b18-4258-a6e7-37fb176bf63f None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:39.157067 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88db9be4-0b18-4258-a6e7-37fb176bf63f None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:39.158048 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88db9be4-0b18-4258-a6e7-37fb176bf63f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:39.177831 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-88db9be4-0b18-4258-a6e7-37fb176bf63f None None] MFA Rules not processed for user `5338bc8c594e4bc9bc41019226bf8d87`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:04:39.178973 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-88db9be4-0b18-4258-a6e7-37fb176bf63f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:04:39.247089 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1159/2320] 10.222.0.64 () {58 vars in 1009 bytes} [Tue Aug 2 10:04:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:04:39.295846 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a20c024-cc8a-474b-81e3-19387de3e6a7 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:39.297385 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-8a20c024-cc8a-474b-81e3-19387de3e6a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:04:39.371632 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a20c024-cc8a-474b-81e3-19387de3e6a7 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:39.376722 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184acecf-2283-41bb-afcc-d92ccaaa5e1d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a371e3df3764a4cbb1b8368aa74bd65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:39.377340 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184acecf-2283-41bb-afcc-d92ccaaa5e1d service swift] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:39.377621 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184acecf-2283-41bb-afcc-d92ccaaa5e1d service swift] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:39.377892 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184acecf-2283-41bb-afcc-d92ccaaa5e1d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:39.386552 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184acecf-2283-41bb-afcc-d92ccaaa5e1d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:39.388519 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184acecf-2283-41bb-afcc-d92ccaaa5e1d service swift] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:39.414275 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1162/2321] 10.222.0.64 () {60 vars in 1384 bytes} [Tue Aug 2 10:04:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 02 10:04:43.010090 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88db9be4-0b18-4258-a6e7-37fb176bf63f None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:43.026304 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-88db9be4-0b18-4258-a6e7-37fb176bf63f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:04:43.106620 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:04:43.179686 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88db9be4-0b18-4258-a6e7-37fb176bf63f None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:43.194955 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4668f93a-b59c-47a2-bf4d-8bfbf72f3973 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99655e1f9ca24472a86cd26b0dbd1fc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:43.197656 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4668f93a-b59c-47a2-bf4d-8bfbf72f3973 service nova] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:43.197656 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4668f93a-b59c-47a2-bf4d-8bfbf72f3973 service nova] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:43.197656 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4668f93a-b59c-47a2-bf4d-8bfbf72f3973 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:43.199153 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-4668f93a-b59c-47a2-bf4d-8bfbf72f3973 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:04:43.269396 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4668f93a-b59c-47a2-bf4d-8bfbf72f3973 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:43.271012 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4668f93a-b59c-47a2-bf4d-8bfbf72f3973 service nova] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:43.294439 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1160/2322] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:04:43 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:04:49.421016 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=83,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:04:52.276553 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184acecf-2283-41bb-afcc-d92ccaaa5e1d service swift] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:52.278010 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-184acecf-2283-41bb-afcc-d92ccaaa5e1d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:04:52.335757 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184acecf-2283-41bb-afcc-d92ccaaa5e1d service swift] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:52.339403 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413e9cf5-6f7c-4013-901b-deb12b184f09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:52.340014 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413e9cf5-6f7c-4013-901b-deb12b184f09 service neutron] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:52.340418 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413e9cf5-6f7c-4013-901b-deb12b184f09 service neutron] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:52.340700 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413e9cf5-6f7c-4013-901b-deb12b184f09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:52.342451 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-413e9cf5-6f7c-4013-901b-deb12b184f09 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:04:52.351703 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413e9cf5-6f7c-4013-901b-deb12b184f09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:52.353171 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413e9cf5-6f7c-4013-901b-deb12b184f09 service neutron] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:52.360946 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1163/2323] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:04:52 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:04:53.277080 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=44 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:04:57.036454 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4668f93a-b59c-47a2-bf4d-8bfbf72f3973 service nova] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:57.037944 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-4668f93a-b59c-47a2-bf4d-8bfbf72f3973 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:04:57.050139 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4668f93a-b59c-47a2-bf4d-8bfbf72f3973 service nova] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:57.053772 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210b863b-8efd-462a-95a3-a4939e95b942 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:57.054373 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210b863b-8efd-462a-95a3-a4939e95b942 service neutron] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:57.054654 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210b863b-8efd-462a-95a3-a4939e95b942 service neutron] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:57.054927 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210b863b-8efd-462a-95a3-a4939e95b942 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:57.056643 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-210b863b-8efd-462a-95a3-a4939e95b942 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:04:57.066002 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210b863b-8efd-462a-95a3-a4939e95b942 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:57.067407 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210b863b-8efd-462a-95a3-a4939e95b942 service neutron] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:57.075518 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1161/2324] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:04:57 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:04:57.195756 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdd608b-94ac-46b5-969a-b79cbb30198b None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:57.196299 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdd608b-94ac-46b5-969a-b79cbb30198b None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:57.196299 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdd608b-94ac-46b5-969a-b79cbb30198b None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:57.198002 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1164/2325] 10.222.0.64 () {54 vars in 921 bytes} [Tue Aug 2 10:04:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:04:57.204670 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d367841-03de-4ffb-8d09-d215b49f09ba None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:57.205030 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d367841-03de-4ffb-8d09-d215b49f09ba None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:57.205373 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d367841-03de-4ffb-8d09-d215b49f09ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:57.225626 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-4d367841-03de-4ffb-8d09-d215b49f09ba None None] MFA Rules not processed for user `99655e1f9ca24472a86cd26b0dbd1fc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:04:57.226930 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-4d367841-03de-4ffb-8d09-d215b49f09ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:04:57.237140 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1162/2326] 10.222.0.64 () {58 vars in 1018 bytes} [Tue Aug 2 10:04:57 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:04:57.248661 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdd608b-94ac-46b5-969a-b79cbb30198b None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:57.250339 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-3fdd608b-94ac-46b5-969a-b79cbb30198b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:04:57.306965 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdd608b-94ac-46b5-969a-b79cbb30198b None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:57.310511 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941f35f8-8afc-4239-95ac-ce432f956419 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04fb1dbe260a4721aaa0bd0493260d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:57.311120 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941f35f8-8afc-4239-95ac-ce432f956419 service glance] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:57.311400 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941f35f8-8afc-4239-95ac-ce432f956419 service glance] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:57.311669 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941f35f8-8afc-4239-95ac-ce432f956419 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:57.322516 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941f35f8-8afc-4239-95ac-ce432f956419 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:57.323958 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941f35f8-8afc-4239-95ac-ce432f956419 service glance] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:57.335989 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1165/2327] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:04:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:04:57.409725 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f26bfc-ed04-4a32-a8c4-b1cf725570d4 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:57.410042 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f26bfc-ed04-4a32-a8c4-b1cf725570d4 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:57.410322 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f26bfc-ed04-4a32-a8c4-b1cf725570d4 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:57.411693 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1163/2328] 10.222.0.64 () {54 vars in 921 bytes} [Tue Aug 2 10:04:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:04:57.416417 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07457b84-3705-44aa-ae33-94ea6b88d142 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:57.416760 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07457b84-3705-44aa-ae33-94ea6b88d142 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:57.417034 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07457b84-3705-44aa-ae33-94ea6b88d142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:57.436615 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-07457b84-3705-44aa-ae33-94ea6b88d142 None None] MFA Rules not processed for user `ec31b188970947efb2ed1d4d9a31cadf`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:04:57.437860 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-07457b84-3705-44aa-ae33-94ea6b88d142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:04:57.444797 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1166/2329] 10.222.0.64 () {58 vars in 1018 bytes} [Tue Aug 2 10:04:57 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:04:57.457293 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f26bfc-ed04-4a32-a8c4-b1cf725570d4 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:57.468811 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f26bfc-ed04-4a32-a8c4-b1cf725570d4 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:57.473524 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850eb47b-39cd-45ce-b9c0-887fca6f8cfe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:57.474123 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850eb47b-39cd-45ce-b9c0-887fca6f8cfe service neutron] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:57.474402 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850eb47b-39cd-45ce-b9c0-887fca6f8cfe service neutron] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:57.474673 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850eb47b-39cd-45ce-b9c0-887fca6f8cfe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:57.481708 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850eb47b-39cd-45ce-b9c0-887fca6f8cfe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:57.483122 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850eb47b-39cd-45ce-b9c0-887fca6f8cfe service neutron] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:57.490926 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1164/2330] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:04:57 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:04:57.572727 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d50af30-2256-4537-9e8a-0368e688894f None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:57.573020 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d50af30-2256-4537-9e8a-0368e688894f None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:57.573301 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d50af30-2256-4537-9e8a-0368e688894f None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:57.574689 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1167/2331] 10.222.0.64 () {54 vars in 921 bytes} [Tue Aug 2 10:04:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:04:57.580204 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ce0de1-b3ad-4f3f-885a-d9e937081b71 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:57.580517 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ce0de1-b3ad-4f3f-885a-d9e937081b71 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:57.580795 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ce0de1-b3ad-4f3f-885a-d9e937081b71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:57.599767 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-d0ce0de1-b3ad-4f3f-885a-d9e937081b71 None None] MFA Rules not processed for user `99655e1f9ca24472a86cd26b0dbd1fc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:04:57.601002 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-d0ce0de1-b3ad-4f3f-885a-d9e937081b71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:04:57.610891 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1165/2332] 10.222.0.64 () {58 vars in 1018 bytes} [Tue Aug 2 10:04:57 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:04:57.621369 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d50af30-2256-4537-9e8a-0368e688894f None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:57.632750 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d50af30-2256-4537-9e8a-0368e688894f None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:57.636411 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6e27df-9b85-4327-af0c-51cb79358822 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:57.637018 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6e27df-9b85-4327-af0c-51cb79358822 service neutron] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:57.637306 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6e27df-9b85-4327-af0c-51cb79358822 service neutron] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:57.637581 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6e27df-9b85-4327-af0c-51cb79358822 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:57.648140 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6e27df-9b85-4327-af0c-51cb79358822 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:57.649607 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6e27df-9b85-4327-af0c-51cb79358822 service neutron] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:57.660117 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1168/2333] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:04:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:04:57.722680 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ce0de1-b3ad-4f3f-885a-d9e937081b71 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:04:57.724062 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-d0ce0de1-b3ad-4f3f-885a-d9e937081b71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:04:57.790766 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ce0de1-b3ad-4f3f-885a-d9e937081b71 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:04:57.796040 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66adccc6-350c-4c7a-8fa6-09019cbf75c2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b770f073c86b4d88837a8b1f796a8b1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:04:57.796873 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66adccc6-350c-4c7a-8fa6-09019cbf75c2 service placement] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:04:57.797156 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66adccc6-350c-4c7a-8fa6-09019cbf75c2 service placement] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:04:57.797433 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66adccc6-350c-4c7a-8fa6-09019cbf75c2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:04:57.799259 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-66adccc6-350c-4c7a-8fa6-09019cbf75c2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:04:57.810484 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66adccc6-350c-4c7a-8fa6-09019cbf75c2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:04:57.812131 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66adccc6-350c-4c7a-8fa6-09019cbf75c2 service placement] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:04:57.822539 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1166/2334] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:04:57 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:05:00.684941 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21107c0-79dc-41b6-8beb-2c245b9c07c9 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:05:00.685426 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21107c0-79dc-41b6-8beb-2c245b9c07c9 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:05:00.685519 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21107c0-79dc-41b6-8beb-2c245b9c07c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:05:00.705742 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-f21107c0-79dc-41b6-8beb-2c245b9c07c9 None None] MFA Rules not processed for user `5338bc8c594e4bc9bc41019226bf8d87`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:05:00.706950 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-f21107c0-79dc-41b6-8beb-2c245b9c07c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:05:00.714158 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1169/2335] 10.222.0.64 () {58 vars in 1009 bytes} [Tue Aug 2 10:05:00 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:05:00.725857 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66adccc6-350c-4c7a-8fa6-09019cbf75c2 service placement] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:05:00.737026 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66adccc6-350c-4c7a-8fa6-09019cbf75c2 service placement] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:05:00.741933 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ac96b8-5bd6-4623-990c-849728ccbfac service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a371e3df3764a4cbb1b8368aa74bd65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:05:00.742544 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ac96b8-5bd6-4623-990c-849728ccbfac service swift] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:05:00.742827 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ac96b8-5bd6-4623-990c-849728ccbfac service swift] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:05:00.743101 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ac96b8-5bd6-4623-990c-849728ccbfac service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:05:00.751992 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ac96b8-5bd6-4623-990c-849728ccbfac service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:05:00.753596 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ac96b8-5bd6-4623-990c-849728ccbfac service swift] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:05:00.761656 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1167/2336] 10.222.0.64 () {60 vars in 1384 bytes} [Tue Aug 2 10:05:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 02 10:05:10.700192 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=64 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:05:10.758400 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=56 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:05:31.012364 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21107c0-79dc-41b6-8beb-2c245b9c07c9 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:05:31.026971 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21107c0-79dc-41b6-8beb-2c245b9c07c9 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:05:31.032508 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d2bb29-d1d4-464e-96bf-a929c1485b8e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99655e1f9ca24472a86cd26b0dbd1fc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:05:31.033172 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d2bb29-d1d4-464e-96bf-a929c1485b8e service nova] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:05:31.034025 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d2bb29-d1d4-464e-96bf-a929c1485b8e service nova] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:05:31.034305 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d2bb29-d1d4-464e-96bf-a929c1485b8e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:05:31.036055 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-46d2bb29-d1d4-464e-96bf-a929c1485b8e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:05:31.048634 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d2bb29-d1d4-464e-96bf-a929c1485b8e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:05:31.050378 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d2bb29-d1d4-464e-96bf-a929c1485b8e service nova] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:05:31.061346 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1170/2337] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:05:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:05:34.643407 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf646361-fa00-4d5e-b847-69043d73ef4e None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:05:34.643966 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf646361-fa00-4d5e-b847-69043d73ef4e None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:05:34.643966 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf646361-fa00-4d5e-b847-69043d73ef4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:05:34.663092 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-cf646361-fa00-4d5e-b847-69043d73ef4e None None] MFA Rules not processed for user `5338bc8c594e4bc9bc41019226bf8d87`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:05:34.664312 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-cf646361-fa00-4d5e-b847-69043d73ef4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:05:34.671401 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1168/2338] 10.222.0.64 () {58 vars in 967 bytes} [Tue Aug 2 10:05:34 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:05:34.683763 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d2bb29-d1d4-464e-96bf-a929c1485b8e service nova] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:05:34.695739 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d2bb29-d1d4-464e-96bf-a929c1485b8e service nova] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:05:34.700638 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbd6b58-993a-4793-91e7-3ea37d798dfb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a371e3df3764a4cbb1b8368aa74bd65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:05:34.701237 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbd6b58-993a-4793-91e7-3ea37d798dfb service swift] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:05:34.701520 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbd6b58-993a-4793-91e7-3ea37d798dfb service swift] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:05:34.701795 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbd6b58-993a-4793-91e7-3ea37d798dfb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:05:34.709582 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbd6b58-993a-4793-91e7-3ea37d798dfb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:05:34.711146 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbd6b58-993a-4793-91e7-3ea37d798dfb service swift] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:05:34.719180 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1171/2339] 10.222.0.64 () {60 vars in 1384 bytes} [Tue Aug 2 10:05:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 02 10:05:44.658262 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=3 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:05:44.715769 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=12 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:05:50.156697 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf646361-fa00-4d5e-b847-69043d73ef4e None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:05:50.169414 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf646361-fa00-4d5e-b847-69043d73ef4e None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:05:50.174804 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be79e64-3651-4277-aaa4-af740615684e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99655e1f9ca24472a86cd26b0dbd1fc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:05:50.175408 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be79e64-3651-4277-aaa4-af740615684e service nova] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:05:50.175690 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be79e64-3651-4277-aaa4-af740615684e service nova] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:05:50.175964 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be79e64-3651-4277-aaa4-af740615684e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:05:50.177721 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-5be79e64-3651-4277-aaa4-af740615684e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:05:50.235768 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be79e64-3651-4277-aaa4-af740615684e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:05:50.237984 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be79e64-3651-4277-aaa4-af740615684e service nova] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:05:50.247049 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1169/2340] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:05:50 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:06:00.242813 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=28 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:06:12.326566 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbd6b58-993a-4793-91e7-3ea37d798dfb service swift] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:06:12.327991 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-adbd6b58-993a-4793-91e7-3ea37d798dfb service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:06:12.343583 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbd6b58-993a-4793-91e7-3ea37d798dfb service swift] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:06:12.349998 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4fb7f8-17bb-43c3-812e-f4d8a20f720a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99655e1f9ca24472a86cd26b0dbd1fc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:06:12.350621 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4fb7f8-17bb-43c3-812e-f4d8a20f720a service nova] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:06:12.350905 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4fb7f8-17bb-43c3-812e-f4d8a20f720a service nova] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:06:12.351183 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4fb7f8-17bb-43c3-812e-f4d8a20f720a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:06:12.352918 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-6a4fb7f8-17bb-43c3-812e-f4d8a20f720a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:06:12.410752 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4fb7f8-17bb-43c3-812e-f4d8a20f720a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:06:12.412909 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4fb7f8-17bb-43c3-812e-f4d8a20f720a service nova] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:06:12.421795 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1172/2341] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:06:12 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:06:21.744392 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55d45d7-1018-48cf-a492-8e3782a9b84e None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:06:21.744974 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55d45d7-1018-48cf-a492-8e3782a9b84e None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:06:21.744974 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55d45d7-1018-48cf-a492-8e3782a9b84e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:06:21.764421 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-f55d45d7-1018-48cf-a492-8e3782a9b84e None None] MFA Rules not processed for user `5bbba5ae48cc4ec39467df736ad1bbad`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:06:21.765627 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-f55d45d7-1018-48cf-a492-8e3782a9b84e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:06:21.772698 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1170/2342] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:06:21 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:06:22.417614 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=28 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:06:27.832472 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4fb7f8-17bb-43c3-812e-f4d8a20f720a service nova] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:06:27.846887 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4fb7f8-17bb-43c3-812e-f4d8a20f720a service nova] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:06:27.854097 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733a08e5-ffc8-4346-b357-e6b068b21952 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99655e1f9ca24472a86cd26b0dbd1fc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:06:27.854739 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733a08e5-ffc8-4346-b357-e6b068b21952 service nova] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:06:27.855082 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733a08e5-ffc8-4346-b357-e6b068b21952 service nova] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:06:27.855361 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733a08e5-ffc8-4346-b357-e6b068b21952 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:06:27.860226 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-733a08e5-ffc8-4346-b357-e6b068b21952 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:06:27.873009 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733a08e5-ffc8-4346-b357-e6b068b21952 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:06:27.874601 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733a08e5-ffc8-4346-b357-e6b068b21952 service nova] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:06:27.885409 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1173/2343] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:06:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:06:31.758930 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=3 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:06:37.880843 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=6 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:06:45.642174 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55d45d7-1018-48cf-a492-8e3782a9b84e None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:06:45.654687 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55d45d7-1018-48cf-a492-8e3782a9b84e None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:06:45.660068 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c125198-2c4e-453f-b786-4e8bc004d50d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99655e1f9ca24472a86cd26b0dbd1fc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:06:45.660701 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c125198-2c4e-453f-b786-4e8bc004d50d service nova] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:06:45.660986 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c125198-2c4e-453f-b786-4e8bc004d50d service nova] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:06:45.661256 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c125198-2c4e-453f-b786-4e8bc004d50d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:06:45.662966 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-8c125198-2c4e-453f-b786-4e8bc004d50d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:06:45.721507 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c125198-2c4e-453f-b786-4e8bc004d50d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:06:45.723012 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c125198-2c4e-453f-b786-4e8bc004d50d service nova] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:06:45.731860 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1171/2344] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:06:45 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:06:47.283320 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733a08e5-ffc8-4346-b357-e6b068b21952 service nova] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:06:47.295404 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733a08e5-ffc8-4346-b357-e6b068b21952 service nova] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:06:47.301095 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2ec8ad-8d2c-43dc-ad8f-73db7db71f31 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99655e1f9ca24472a86cd26b0dbd1fc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:06:47.302832 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2ec8ad-8d2c-43dc-ad8f-73db7db71f31 service nova] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:06:47.302832 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2ec8ad-8d2c-43dc-ad8f-73db7db71f31 service nova] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:06:47.302832 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2ec8ad-8d2c-43dc-ad8f-73db7db71f31 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:06:47.304325 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-0c2ec8ad-8d2c-43dc-ad8f-73db7db71f31 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:06:47.363430 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2ec8ad-8d2c-43dc-ad8f-73db7db71f31 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:06:47.365173 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2ec8ad-8d2c-43dc-ad8f-73db7db71f31 service nova] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:06:47.374447 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1174/2345] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:06:47 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:06:51.518278 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c125198-2c4e-453f-b786-4e8bc004d50d service nova] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:06:51.545290 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c125198-2c4e-453f-b786-4e8bc004d50d service nova] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:06:51.563020 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bde8a8-ffde-43f6-9823-cf315ce6a0a1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99655e1f9ca24472a86cd26b0dbd1fc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:06:51.563674 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bde8a8-ffde-43f6-9823-cf315ce6a0a1 service nova] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:06:51.563965 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bde8a8-ffde-43f6-9823-cf315ce6a0a1 service nova] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:06:51.564266 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bde8a8-ffde-43f6-9823-cf315ce6a0a1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:06:51.566267 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-18bde8a8-ffde-43f6-9823-cf315ce6a0a1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:06:51.581418 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bde8a8-ffde-43f6-9823-cf315ce6a0a1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:06:51.583695 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bde8a8-ffde-43f6-9823-cf315ce6a0a1 service nova] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:06:51.596718 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1172/2346] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:06:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:06:53.393697 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0e0544-20b2-4ba3-bd6d-3ec310aa5b9a None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:06:53.394291 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0e0544-20b2-4ba3-bd6d-3ec310aa5b9a None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:06:53.394291 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0e0544-20b2-4ba3-bd6d-3ec310aa5b9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:06:53.415478 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-ad0e0544-20b2-4ba3-bd6d-3ec310aa5b9a None None] MFA Rules not processed for user `7739a00c6524467d8fd139183c6667cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:06:53.416816 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-ad0e0544-20b2-4ba3-bd6d-3ec310aa5b9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:06:53.423873 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1175/2347] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:06:53 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:06:53.772708 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bde8a8-ffde-43f6-9823-cf315ce6a0a1 service nova] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:06:53.779332 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bde8a8-ffde-43f6-9823-cf315ce6a0a1 service nova] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:06:53.782876 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a36d4da-dd07-476c-bbf1-ebbdc47e4987 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:06:53.783480 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a36d4da-dd07-476c-bbf1-ebbdc47e4987 service neutron] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:06:53.783759 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a36d4da-dd07-476c-bbf1-ebbdc47e4987 service neutron] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:06:53.784031 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a36d4da-dd07-476c-bbf1-ebbdc47e4987 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:06:53.785916 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-2a36d4da-dd07-476c-bbf1-ebbdc47e4987 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:06:53.795446 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a36d4da-dd07-476c-bbf1-ebbdc47e4987 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:06:53.796911 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a36d4da-dd07-476c-bbf1-ebbdc47e4987 service neutron] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:06:53.804781 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1173/2348] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:06:53 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:07:03.062050 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3bf096-3426-49f7-93af-2bcd06f3234c None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:07:03.062552 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3bf096-3426-49f7-93af-2bcd06f3234c None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:07:03.062646 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3bf096-3426-49f7-93af-2bcd06f3234c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:07:03.081496 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-ae3bf096-3426-49f7-93af-2bcd06f3234c None None] MFA Rules not processed for user `dfe32e33711e42118f3ff5021762604f`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:07:03.082682 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-ae3bf096-3426-49f7-93af-2bcd06f3234c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:07:03.089931 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1176/2349] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:07:03 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:07:03.801222 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=40 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:07:13.076372 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=34 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:07:20.671804 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48b6d96-0f20-4551-b684-f30d32620383 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:07:20.672777 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48b6d96-0f20-4551-b684-f30d32620383 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:07:20.673252 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48b6d96-0f20-4551-b684-f30d32620383 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:07:20.694299 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-e48b6d96-0f20-4551-b684-f30d32620383 None None] MFA Rules not processed for user `c94f607a834f43e7a0073fe27d265901`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:07:20.695509 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-e48b6d96-0f20-4551-b684-f30d32620383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:07:20.703474 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1174/2350] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:07:20 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:07:30.687944 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=3 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:07:37.965102 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3bf096-3426-49f7-93af-2bcd06f3234c None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:07:38.026560 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48b6d96-0f20-4551-b684-f30d32620383 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:07:38.044887 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3bf096-3426-49f7-93af-2bcd06f3234c None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:07:38.053426 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d9e5e4-985d-434b-beb8-63385a463966 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99655e1f9ca24472a86cd26b0dbd1fc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:07:38.054395 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d9e5e4-985d-434b-beb8-63385a463966 service nova] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:07:38.054879 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d9e5e4-985d-434b-beb8-63385a463966 service nova] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:07:38.055545 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d9e5e4-985d-434b-beb8-63385a463966 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:07:38.058458 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-65d9e5e4-985d-434b-beb8-63385a463966 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:07:38.090817 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48b6d96-0f20-4551-b684-f30d32620383 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:07:38.096322 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102ba793-8112-4893-b22b-d363ab27a649 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b770f073c86b4d88837a8b1f796a8b1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:07:38.097092 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102ba793-8112-4893-b22b-d363ab27a649 service placement] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:07:38.097470 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102ba793-8112-4893-b22b-d363ab27a649 service placement] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:07:38.098057 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102ba793-8112-4893-b22b-d363ab27a649 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:07:38.100153 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-102ba793-8112-4893-b22b-d363ab27a649 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:07:38.132753 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102ba793-8112-4893-b22b-d363ab27a649 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:07:38.134696 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102ba793-8112-4893-b22b-d363ab27a649 service placement] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:07:38.147365 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1175/2351] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:07:38 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:07:38.159177 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d9e5e4-985d-434b-beb8-63385a463966 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:07:38.160978 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d9e5e4-985d-434b-beb8-63385a463966 service nova] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:07:38.172141 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1177/2352] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:07:37 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:07:40.515637 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e595f30c-c45c-4034-b8b0-3248cbf596d7 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:07:40.516096 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e595f30c-c45c-4034-b8b0-3248cbf596d7 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:07:40.516211 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e595f30c-c45c-4034-b8b0-3248cbf596d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:07:40.545458 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-e595f30c-c45c-4034-b8b0-3248cbf596d7 None None] MFA Rules not processed for user `5338bc8c594e4bc9bc41019226bf8d87`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:07:40.546664 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-e595f30c-c45c-4034-b8b0-3248cbf596d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:07:40.553804 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1176/2353] 10.222.0.64 () {58 vars in 967 bytes} [Tue Aug 2 10:07:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:07:40.601184 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d9e5e4-985d-434b-beb8-63385a463966 service nova] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:07:40.613141 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d9e5e4-985d-434b-beb8-63385a463966 service nova] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:07:40.618155 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8c6aa9-6f6d-48ed-a270-62925cb7d0cb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a371e3df3764a4cbb1b8368aa74bd65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:07:40.618767 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8c6aa9-6f6d-48ed-a270-62925cb7d0cb service swift] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:07:40.619057 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8c6aa9-6f6d-48ed-a270-62925cb7d0cb service swift] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:07:40.619336 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8c6aa9-6f6d-48ed-a270-62925cb7d0cb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:07:40.628148 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8c6aa9-6f6d-48ed-a270-62925cb7d0cb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:07:40.629803 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8c6aa9-6f6d-48ed-a270-62925cb7d0cb service swift] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:07:40.638000 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1178/2354] 10.222.0.64 () {60 vars in 1384 bytes} [Tue Aug 2 10:07:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 02 10:07:41.012740 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e595f30c-c45c-4034-b8b0-3248cbf596d7 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:07:41.032032 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e595f30c-c45c-4034-b8b0-3248cbf596d7 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:07:41.035397 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccd4135-fad0-4de5-a81f-1f77107a08a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:07:41.036031 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccd4135-fad0-4de5-a81f-1f77107a08a3 service neutron] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:07:41.036363 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccd4135-fad0-4de5-a81f-1f77107a08a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:07:41.036665 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccd4135-fad0-4de5-a81f-1f77107a08a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:07:41.038429 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-fccd4135-fad0-4de5-a81f-1f77107a08a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:07:41.124435 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccd4135-fad0-4de5-a81f-1f77107a08a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:07:41.125883 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccd4135-fad0-4de5-a81f-1f77107a08a3 service neutron] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:07:41.145192 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1177/2355] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:07:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:07:47.783321 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8c6aa9-6f6d-48ed-a270-62925cb7d0cb service swift] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:07:47.800720 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8c6aa9-6f6d-48ed-a270-62925cb7d0cb service swift] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:07:47.806249 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b4c08b-5e77-4ade-beae-d773d80e085b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b770f073c86b4d88837a8b1f796a8b1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:07:47.806925 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b4c08b-5e77-4ade-beae-d773d80e085b service placement] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:07:47.807423 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b4c08b-5e77-4ade-beae-d773d80e085b service placement] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:07:47.808444 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b4c08b-5e77-4ade-beae-d773d80e085b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:07:47.810538 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-06b4c08b-5e77-4ade-beae-d773d80e085b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:07:47.824061 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b4c08b-5e77-4ade-beae-d773d80e085b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:07:47.825735 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b4c08b-5e77-4ade-beae-d773d80e085b service placement] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:07:47.837627 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1179/2356] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:07:47 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:07:47.862355 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccd4135-fad0-4de5-a81f-1f77107a08a3 service neutron] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:07:47.863899 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-fccd4135-fad0-4de5-a81f-1f77107a08a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:07:47.878147 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccd4135-fad0-4de5-a81f-1f77107a08a3 service neutron] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:07:47.896769 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24afebc2-efa7-401d-9816-e5dc235b3a4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:07:47.897538 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24afebc2-efa7-401d-9816-e5dc235b3a4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:07:47.898023 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24afebc2-efa7-401d-9816-e5dc235b3a4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:07:47.898483 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24afebc2-efa7-401d-9816-e5dc235b3a4b admin admin] PATH_INFO: `/v3/users/3f9506226c4846628d32aadb0f6b86be` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:07:47.899810 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24afebc2-efa7-401d-9816-e5dc235b3a4b admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f9506226c4846628d32aadb0f6b86be)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:07:47.902635 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24afebc2-efa7-401d-9816-e5dc235b3a4b admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:07:47.951715 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:07:47.953272 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-24afebc2-efa7-401d-9816-e5dc235b3a4b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f9506226c4846628d32aadb0f6b86be'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:07:47.961595 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-24afebc2-efa7-401d-9816-e5dc235b3a4b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f9506226c4846628d32aadb0f6b86be'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:07:47.962286 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-24afebc2-efa7-401d-9816-e5dc235b3a4b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f9506226c4846628d32aadb0f6b86be'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:07:47.970388 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-24afebc2-efa7-401d-9816-e5dc235b3a4b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f9506226c4846628d32aadb0f6b86be'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:07:47.982855 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-24afebc2-efa7-401d-9816-e5dc235b3a4b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f9506226c4846628d32aadb0f6b86be'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:07:47.989280 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1178/2357] 10.222.0.64 () {58 vars in 1224 bytes} [Tue Aug 2 10:07:47 2022] DELETE /identity/v3/users/3f9506226c4846628d32aadb0f6b86be => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:07:48.172628 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b4c08b-5e77-4ade-beae-d773d80e085b service placement] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:07:48.184911 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b4c08b-5e77-4ade-beae-d773d80e085b service placement] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:07:48.246216 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec14bc6b-b9ae-468e-9cae-5eb377727884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:07:48.246865 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec14bc6b-b9ae-468e-9cae-5eb377727884 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:07:48.247148 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec14bc6b-b9ae-468e-9cae-5eb377727884 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:07:48.247424 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec14bc6b-b9ae-468e-9cae-5eb377727884 admin admin] PATH_INFO: `/v3/projects/52ea66f641ea4e1f877d88f5ddc4c226` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:07:48.248625 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec14bc6b-b9ae-468e-9cae-5eb377727884 admin admin] RBAC: Authorizing `identity:delete_project(project_id=52ea66f641ea4e1f877d88f5ddc4c226)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:07:48.250546 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec14bc6b-b9ae-468e-9cae-5eb377727884 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:07:48.262751 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:07:48.304045 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-ec14bc6b-b9ae-468e-9cae-5eb377727884 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52ea66f641ea4e1f877d88f5ddc4c226', 'request_id': 'req-ec14bc6b-b9ae-468e-9cae-5eb377727884'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:07:48.310740 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-ec14bc6b-b9ae-468e-9cae-5eb377727884 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52ea66f641ea4e1f877d88f5ddc4c226', 'request_id': 'req-ec14bc6b-b9ae-468e-9cae-5eb377727884'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:07:48.317816 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-ec14bc6b-b9ae-468e-9cae-5eb377727884 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52ea66f641ea4e1f877d88f5ddc4c226', 'request_id': 'req-ec14bc6b-b9ae-468e-9cae-5eb377727884'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:07:48.330796 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-ec14bc6b-b9ae-468e-9cae-5eb377727884 admin admin] The token cache is being invalidate because project 52ea66f641ea4e1f877d88f5ddc4c226 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80153) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:07:48.331094 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-ec14bc6b-b9ae-468e-9cae-5eb377727884 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:07:48.331722 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-ec14bc6b-b9ae-468e-9cae-5eb377727884 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:07:48.333284 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1180/2358] 10.222.0.64 () {58 vars in 1233 bytes} [Tue Aug 2 10:07:48 2022] DELETE /identity/v3/projects/52ea66f641ea4e1f877d88f5ddc4c226 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:07:48.337314 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24afebc2-efa7-401d-9816-e5dc235b3a4b admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:07:48.338763 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-24afebc2-efa7-401d-9816-e5dc235b3a4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:07:48.351902 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24afebc2-efa7-401d-9816-e5dc235b3a4b admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:07:48.415287 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a2d236-c1eb-45fe-a744-b45a84e0c50d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:07:48.415844 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a2d236-c1eb-45fe-a744-b45a84e0c50d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:07:48.416131 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a2d236-c1eb-45fe-a744-b45a84e0c50d admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:07:48.416470 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a2d236-c1eb-45fe-a744-b45a84e0c50d admin admin] PATH_INFO: `/v3/users/8026b988ac8d4342b0d3fd2ccfae915c` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:07:48.417666 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a2d236-c1eb-45fe-a744-b45a84e0c50d admin admin] RBAC: Authorizing `identity:delete_user(user_id=8026b988ac8d4342b0d3fd2ccfae915c)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:07:48.419706 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a2d236-c1eb-45fe-a744-b45a84e0c50d admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:07:48.470230 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-67a2d236-c1eb-45fe-a744-b45a84e0c50d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8026b988ac8d4342b0d3fd2ccfae915c'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:07:48.477270 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-67a2d236-c1eb-45fe-a744-b45a84e0c50d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8026b988ac8d4342b0d3fd2ccfae915c'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:07:48.478008 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-67a2d236-c1eb-45fe-a744-b45a84e0c50d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8026b988ac8d4342b0d3fd2ccfae915c'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:07:48.486662 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-67a2d236-c1eb-45fe-a744-b45a84e0c50d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8026b988ac8d4342b0d3fd2ccfae915c'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:07:48.504703 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-67a2d236-c1eb-45fe-a744-b45a84e0c50d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8026b988ac8d4342b0d3fd2ccfae915c'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:07:48.511576 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1179/2359] 10.222.0.64 () {58 vars in 1224 bytes} [Tue Aug 2 10:07:48 2022] DELETE /identity/v3/users/8026b988ac8d4342b0d3fd2ccfae915c => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:07:48.695321 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec14bc6b-b9ae-468e-9cae-5eb377727884 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:07:48.697855 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-ec14bc6b-b9ae-468e-9cae-5eb377727884 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:07:48.714694 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec14bc6b-b9ae-468e-9cae-5eb377727884 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:07:48.780765 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dc9de0-919b-457a-ac0d-21e7d5a1e909 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:07:48.781421 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dc9de0-919b-457a-ac0d-21e7d5a1e909 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:07:48.781711 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dc9de0-919b-457a-ac0d-21e7d5a1e909 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:07:48.781992 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dc9de0-919b-457a-ac0d-21e7d5a1e909 admin admin] PATH_INFO: `/v3/projects/e88d8c3ba87e4a4db25439b1f266fe23` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:07:48.783233 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dc9de0-919b-457a-ac0d-21e7d5a1e909 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e88d8c3ba87e4a4db25439b1f266fe23)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:07:48.786886 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dc9de0-919b-457a-ac0d-21e7d5a1e909 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:07:48.846802 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-08dc9de0-919b-457a-ac0d-21e7d5a1e909 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e88d8c3ba87e4a4db25439b1f266fe23', 'request_id': 'req-08dc9de0-919b-457a-ac0d-21e7d5a1e909'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:07:48.856240 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-08dc9de0-919b-457a-ac0d-21e7d5a1e909 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e88d8c3ba87e4a4db25439b1f266fe23', 'request_id': 'req-08dc9de0-919b-457a-ac0d-21e7d5a1e909'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:07:48.862857 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-08dc9de0-919b-457a-ac0d-21e7d5a1e909 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e88d8c3ba87e4a4db25439b1f266fe23', 'request_id': 'req-08dc9de0-919b-457a-ac0d-21e7d5a1e909'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:07:48.876188 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-08dc9de0-919b-457a-ac0d-21e7d5a1e909 admin admin] The token cache is being invalidate because project e88d8c3ba87e4a4db25439b1f266fe23 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80153) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:07:48.876512 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-08dc9de0-919b-457a-ac0d-21e7d5a1e909 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:07:48.877130 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-08dc9de0-919b-457a-ac0d-21e7d5a1e909 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:07:48.878709 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1181/2360] 10.222.0.64 () {58 vars in 1233 bytes} [Tue Aug 2 10:07:48 2022] DELETE /identity/v3/projects/e88d8c3ba87e4a4db25439b1f266fe23 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:07:58.507416 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=66,keystone:INSERT=2,keystone:DELETE=13 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:07:58.873203 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=70,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:07:59.963460 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef03fbce-07b6-449a-a99a-4026ac331c2b None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:07:59.963964 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef03fbce-07b6-449a-a99a-4026ac331c2b None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:07:59.964063 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef03fbce-07b6-449a-a99a-4026ac331c2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:07:59.983191 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-ef03fbce-07b6-449a-a99a-4026ac331c2b None None] MFA Rules not processed for user `982a01f0d3a5459a9add3f6f23db5db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:07:59.984449 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-ef03fbce-07b6-449a-a99a-4026ac331c2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:08:00.038736 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1180/2361] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:07:59 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:08:10.032110 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=25 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:08:11.921640 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dc9de0-919b-457a-ac0d-21e7d5a1e909 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:08:11.923358 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-08dc9de0-919b-457a-ac0d-21e7d5a1e909 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:08:12.010079 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dc9de0-919b-457a-ac0d-21e7d5a1e909 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:08:12.016064 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a96d85-65c0-45c7-a917-5fb5b27f59f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:08:12.016942 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a96d85-65c0-45c7-a917-5fb5b27f59f9 service neutron] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:08:12.017295 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a96d85-65c0-45c7-a917-5fb5b27f59f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:08:12.017594 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a96d85-65c0-45c7-a917-5fb5b27f59f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:08:12.019909 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-09a96d85-65c0-45c7-a917-5fb5b27f59f9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:08:12.097613 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a96d85-65c0-45c7-a917-5fb5b27f59f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:08:12.099079 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a96d85-65c0-45c7-a917-5fb5b27f59f9 service neutron] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:08:12.109685 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1182/2362] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:08:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:08:16.133153 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef03fbce-07b6-449a-a99a-4026ac331c2b None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:08:16.135134 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-ef03fbce-07b6-449a-a99a-4026ac331c2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:08:16.154236 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef03fbce-07b6-449a-a99a-4026ac331c2b None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:08:16.161592 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c1ad04-a53e-4767-84ea-252ab844c9f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:08:16.163049 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c1ad04-a53e-4767-84ea-252ab844c9f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:08:16.164178 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c1ad04-a53e-4767-84ea-252ab844c9f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:08:16.164959 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c1ad04-a53e-4767-84ea-252ab844c9f8 admin admin] PATH_INFO: `/v3/users/5bbba5ae48cc4ec39467df736ad1bbad` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:08:16.167093 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c1ad04-a53e-4767-84ea-252ab844c9f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5bbba5ae48cc4ec39467df736ad1bbad)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:08:16.171860 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c1ad04-a53e-4767-84ea-252ab844c9f8 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:08:16.229959 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-05c1ad04-a53e-4767-84ea-252ab844c9f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5bbba5ae48cc4ec39467df736ad1bbad'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:08:16.237410 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-05c1ad04-a53e-4767-84ea-252ab844c9f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5bbba5ae48cc4ec39467df736ad1bbad'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:08:16.238195 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-05c1ad04-a53e-4767-84ea-252ab844c9f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5bbba5ae48cc4ec39467df736ad1bbad'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:08:16.247611 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-05c1ad04-a53e-4767-84ea-252ab844c9f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5bbba5ae48cc4ec39467df736ad1bbad'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:08:16.261507 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-05c1ad04-a53e-4767-84ea-252ab844c9f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5bbba5ae48cc4ec39467df736ad1bbad'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:08:16.270096 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1181/2363] 10.222.0.64 () {58 vars in 1224 bytes} [Tue Aug 2 10:08:16 2022] DELETE /identity/v3/users/5bbba5ae48cc4ec39467df736ad1bbad => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:08:16.457909 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a96d85-65c0-45c7-a917-5fb5b27f59f9 service neutron] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:08:16.470283 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a96d85-65c0-45c7-a917-5fb5b27f59f9 service neutron] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:08:16.557343 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601ded06-0260-43e3-b3a3-eac064395e0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:08:16.558516 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601ded06-0260-43e3-b3a3-eac064395e0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:08:16.558899 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601ded06-0260-43e3-b3a3-eac064395e0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:08:16.561796 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601ded06-0260-43e3-b3a3-eac064395e0e admin admin] PATH_INFO: `/v3/projects/ec62d727071640d8b76345246bb7be13` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:08:16.564046 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601ded06-0260-43e3-b3a3-eac064395e0e admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec62d727071640d8b76345246bb7be13)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:08:16.568355 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601ded06-0260-43e3-b3a3-eac064395e0e admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:08:16.621530 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:08:16.629430 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-601ded06-0260-43e3-b3a3-eac064395e0e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec62d727071640d8b76345246bb7be13', 'request_id': 'req-601ded06-0260-43e3-b3a3-eac064395e0e'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:08:16.642156 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-601ded06-0260-43e3-b3a3-eac064395e0e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec62d727071640d8b76345246bb7be13', 'request_id': 'req-601ded06-0260-43e3-b3a3-eac064395e0e'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:08:16.643525 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-601ded06-0260-43e3-b3a3-eac064395e0e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec62d727071640d8b76345246bb7be13', 'request_id': 'req-601ded06-0260-43e3-b3a3-eac064395e0e'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:08:16.659160 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-601ded06-0260-43e3-b3a3-eac064395e0e admin admin] The token cache is being invalidate because project ec62d727071640d8b76345246bb7be13 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80153) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:08:16.659549 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-601ded06-0260-43e3-b3a3-eac064395e0e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:08:16.660380 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-601ded06-0260-43e3-b3a3-eac064395e0e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:08:16.662671 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1183/2364] 10.222.0.64 () {58 vars in 1233 bytes} [Tue Aug 2 10:08:16 2022] DELETE /identity/v3/projects/ec62d727071640d8b76345246bb7be13 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:08:16.668384 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0bdd4b-9112-4c86-ac4b-80f4b1df6ff8 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:08:16.668823 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0bdd4b-9112-4c86-ac4b-80f4b1df6ff8 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:08:16.669675 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0bdd4b-9112-4c86-ac4b-80f4b1df6ff8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:08:16.747044 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-9a0bdd4b-9112-4c86-ac4b-80f4b1df6ff8 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:08:16.749457 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-9a0bdd4b-9112-4c86-ac4b-80f4b1df6ff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:08:16.851233 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1182/2365] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:08:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:08:16.860093 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b552220d-3f0a-4a6c-afbb-108de907c902 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:08:16.860413 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b552220d-3f0a-4a6c-afbb-108de907c902 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:08:16.860695 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b552220d-3f0a-4a6c-afbb-108de907c902 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:08:16.884289 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-b552220d-3f0a-4a6c-afbb-108de907c902 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:08:16.886191 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-b552220d-3f0a-4a6c-afbb-108de907c902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:08:16.900339 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1184/2366] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:08:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:08:16.904938 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0bdd4b-9112-4c86-ac4b-80f4b1df6ff8 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:08:16.920059 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0bdd4b-9112-4c86-ac4b-80f4b1df6ff8 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:08:16.925230 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dda124-e9cd-4870-8270-ffcaa1dfa393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:08:16.926211 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dda124-e9cd-4870-8270-ffcaa1dfa393 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:08:16.926631 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dda124-e9cd-4870-8270-ffcaa1dfa393 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:08:16.926981 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dda124-e9cd-4870-8270-ffcaa1dfa393 admin admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:08:16.927782 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dda124-e9cd-4870-8270-ffcaa1dfa393 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:08:16.928165 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dda124-e9cd-4870-8270-ffcaa1dfa393 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:08:16.930023 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dda124-e9cd-4870-8270-ffcaa1dfa393 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:08:16.948925 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1183/2367] 10.222.0.64 () {58 vars in 1153 bytes} [Tue Aug 2 10:08:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:08:16.954285 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b552220d-3f0a-4a6c-afbb-108de907c902 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:08:16.963916 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b552220d-3f0a-4a6c-afbb-108de907c902 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:08:16.969896 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817daa3f-ea43-492b-a174-55e463b57755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:08:16.970652 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817daa3f-ea43-492b-a174-55e463b57755 admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:08:16.970968 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817daa3f-ea43-492b-a174-55e463b57755 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:08:16.971250 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817daa3f-ea43-492b-a174-55e463b57755 admin admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:08:16.972130 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817daa3f-ea43-492b-a174-55e463b57755 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:08:16.974300 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817daa3f-ea43-492b-a174-55e463b57755 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:08:17.010585 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1185/2368] 10.222.0.64 () {60 vars in 1153 bytes} [Tue Aug 2 10:08:16 2022] POST /identity/v3/projects => generated 365 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:08:17.015655 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dda124-e9cd-4870-8270-ffcaa1dfa393 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:08:17.027239 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dda124-e9cd-4870-8270-ffcaa1dfa393 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:08:17.057618 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:08:17.097052 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1ca428-c82b-4893-ac59-015d19b51e90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:08:17.097796 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1ca428-c82b-4893-ac59-015d19b51e90 admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:08:17.098115 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1ca428-c82b-4893-ac59-015d19b51e90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:08:17.098451 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1ca428-c82b-4893-ac59-015d19b51e90 admin admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:08:17.099188 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1ca428-c82b-4893-ac59-015d19b51e90 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:08:17.101265 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1ca428-c82b-4893-ac59-015d19b51e90 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:08:17.124780 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1184/2369] 10.222.0.64 () {60 vars in 1144 bytes} [Tue Aug 2 10:08:17 2022] POST /identity/v3/users => generated 283 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:08:17.129580 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817daa3f-ea43-492b-a174-55e463b57755 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:08:17.141398 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817daa3f-ea43-492b-a174-55e463b57755 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:08:17.147034 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1693018d-28ca-4a18-8341-bb457a92c78a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:08:17.147716 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1693018d-28ca-4a18-8341-bb457a92c78a admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:08:17.148002 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1693018d-28ca-4a18-8341-bb457a92c78a admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:08:17.148269 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1693018d-28ca-4a18-8341-bb457a92c78a admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:08:17.149040 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1693018d-28ca-4a18-8341-bb457a92c78a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:08:17.150648 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1693018d-28ca-4a18-8341-bb457a92c78a admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:08:17.161714 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1186/2370] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:08:17 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:08:17.166068 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1ca428-c82b-4893-ac59-015d19b51e90 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:08:17.176716 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1ca428-c82b-4893-ac59-015d19b51e90 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:08:17.181452 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feae5a75-a47d-4b16-9b4d-db0e20a67649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:08:17.182070 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feae5a75-a47d-4b16-9b4d-db0e20a67649 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:08:17.182352 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feae5a75-a47d-4b16-9b4d-db0e20a67649 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:08:17.182627 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feae5a75-a47d-4b16-9b4d-db0e20a67649 admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:08:17.183323 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feae5a75-a47d-4b16-9b4d-db0e20a67649 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:08:17.185450 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feae5a75-a47d-4b16-9b4d-db0e20a67649 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:08:17.197058 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1185/2371] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:08:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:08:17.202170 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1693018d-28ca-4a18-8341-bb457a92c78a admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:08:17.212283 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1693018d-28ca-4a18-8341-bb457a92c78a admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:08:17.217328 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccf1caa-4cf8-4b1d-b4e8-278533f17d01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:08:17.218035 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccf1caa-4cf8-4b1d-b4e8-278533f17d01 admin admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:08:17.218354 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccf1caa-4cf8-4b1d-b4e8-278533f17d01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:08:17.218634 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccf1caa-4cf8-4b1d-b4e8-278533f17d01 admin admin] PATH_INFO: `/v3/projects/75626068a7784c488e8d434a95194ebb/users/23c6192b0ffa44c5a6099204719101d8/roles/66c1074780954f8e87ed11a5c4e8c36a` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:08:17.235812 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccf1caa-4cf8-4b1d-b4e8-278533f17d01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=75626068a7784c488e8d434a95194ebb, user_id=23c6192b0ffa44c5a6099204719101d8, role_id=66c1074780954f8e87ed11a5c4e8c36a)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:08:17.238516 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccf1caa-4cf8-4b1d-b4e8-278533f17d01 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:08:17.252909 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1187/2372] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 10:08:17 2022] PUT /identity/v3/projects/75626068a7784c488e8d434a95194ebb/users/23c6192b0ffa44c5a6099204719101d8/roles/66c1074780954f8e87ed11a5c4e8c36a => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:08:17.264330 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9e41b1-cb63-412b-b715-0d01a69ed7ae None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:08:17.264654 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9e41b1-cb63-412b-b715-0d01a69ed7ae None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:08:17.264972 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9e41b1-cb63-412b-b715-0d01a69ed7ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:08:17.295103 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-6c9e41b1-cb63-412b-b715-0d01a69ed7ae None None] MFA Rules not processed for user `23c6192b0ffa44c5a6099204719101d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:08:17.296352 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-6c9e41b1-cb63-412b-b715-0d01a69ed7ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:08:17.364975 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1186/2373] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:08:17 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:08:17.376392 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccf1caa-4cf8-4b1d-b4e8-278533f17d01 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:08:17.378211 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-fccf1caa-4cf8-4b1d-b4e8-278533f17d01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:08:17.455350 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccf1caa-4cf8-4b1d-b4e8-278533f17d01 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:08:17.459430 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956a54ee-e81c-4dc7-9f32-ab7b4e0cffc5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d7de62b9ee4039bdc79627ac7fb499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:08:17.460084 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956a54ee-e81c-4dc7-9f32-ab7b4e0cffc5 service cinder] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:08:17.460413 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956a54ee-e81c-4dc7-9f32-ab7b4e0cffc5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:08:17.460735 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956a54ee-e81c-4dc7-9f32-ab7b4e0cffc5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:08:17.469626 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956a54ee-e81c-4dc7-9f32-ab7b4e0cffc5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:08:17.471073 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956a54ee-e81c-4dc7-9f32-ab7b4e0cffc5 service cinder] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:08:17.479765 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1188/2374] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:08:17 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:08:27.295562 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9e41b1-cb63-412b-b715-0d01a69ed7ae None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:08:27.391984 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9e41b1-cb63-412b-b715-0d01a69ed7ae None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:08:27.398645 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12153c0f-248e-4b04-b4d9-4642eed6d23f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:08:27.399444 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12153c0f-248e-4b04-b4d9-4642eed6d23f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:08:27.399825 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12153c0f-248e-4b04-b4d9-4642eed6d23f admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:08:27.400355 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12153c0f-248e-4b04-b4d9-4642eed6d23f admin admin] PATH_INFO: `/v3/users/23c6192b0ffa44c5a6099204719101d8` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:08:27.401744 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12153c0f-248e-4b04-b4d9-4642eed6d23f admin admin] RBAC: Authorizing `identity:delete_user(user_id=23c6192b0ffa44c5a6099204719101d8)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:08:27.405796 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12153c0f-248e-4b04-b4d9-4642eed6d23f admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:08:27.467051 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:08:27.475912 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=71,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:08:27.477184 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-12153c0f-248e-4b04-b4d9-4642eed6d23f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23c6192b0ffa44c5a6099204719101d8'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:08:27.487259 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-12153c0f-248e-4b04-b4d9-4642eed6d23f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23c6192b0ffa44c5a6099204719101d8'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:08:27.492483 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-12153c0f-248e-4b04-b4d9-4642eed6d23f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23c6192b0ffa44c5a6099204719101d8'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:08:27.503024 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-12153c0f-248e-4b04-b4d9-4642eed6d23f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23c6192b0ffa44c5a6099204719101d8'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:08:27.519478 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-12153c0f-248e-4b04-b4d9-4642eed6d23f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23c6192b0ffa44c5a6099204719101d8'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:08:27.532169 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1187/2375] 10.222.0.64 () {58 vars in 1224 bytes} [Tue Aug 2 10:08:27 2022] DELETE /identity/v3/users/23c6192b0ffa44c5a6099204719101d8 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:08:27.537865 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b35b60-9a45-4e67-b43a-d49dc3a740bf None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:08:27.538157 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b35b60-9a45-4e67-b43a-d49dc3a740bf None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:08:27.538443 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b35b60-9a45-4e67-b43a-d49dc3a740bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:08:27.561798 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-82b35b60-9a45-4e67-b43a-d49dc3a740bf None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:08:27.563259 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-82b35b60-9a45-4e67-b43a-d49dc3a740bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:08:27.632466 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1189/2376] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:08:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:08:27.643120 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12153c0f-248e-4b04-b4d9-4642eed6d23f admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:08:27.644586 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-12153c0f-248e-4b04-b4d9-4642eed6d23f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:08:27.710452 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12153c0f-248e-4b04-b4d9-4642eed6d23f admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:08:27.714180 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d7dcc3-14d7-43ab-bba6-fd7505a27e7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:08:27.714842 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d7dcc3-14d7-43ab-bba6-fd7505a27e7b service neutron] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:08:27.715130 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d7dcc3-14d7-43ab-bba6-fd7505a27e7b service neutron] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:08:27.715413 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d7dcc3-14d7-43ab-bba6-fd7505a27e7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:08:27.726279 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d7dcc3-14d7-43ab-bba6-fd7505a27e7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:08:27.727753 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d7dcc3-14d7-43ab-bba6-fd7505a27e7b service neutron] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:08:27.738556 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1188/2377] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:08:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:08:28.040591 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b35b60-9a45-4e67-b43a-d49dc3a740bf None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:08:28.042427 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-82b35b60-9a45-4e67-b43a-d49dc3a740bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:08:28.058029 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b35b60-9a45-4e67-b43a-d49dc3a740bf None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:08:28.063464 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ee70af-de39-414e-a3ea-501e93130154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:08:28.064763 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ee70af-de39-414e-a3ea-501e93130154 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:08:28.065052 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ee70af-de39-414e-a3ea-501e93130154 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:08:28.065334 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ee70af-de39-414e-a3ea-501e93130154 admin admin] PATH_INFO: `/v3/projects/75626068a7784c488e8d434a95194ebb` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:08:28.066634 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ee70af-de39-414e-a3ea-501e93130154 admin admin] RBAC: Authorizing `identity:delete_project(project_id=75626068a7784c488e8d434a95194ebb)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:08:28.068674 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ee70af-de39-414e-a3ea-501e93130154 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:08:28.130266 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-01ee70af-de39-414e-a3ea-501e93130154 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75626068a7784c488e8d434a95194ebb', 'request_id': 'req-01ee70af-de39-414e-a3ea-501e93130154'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:08:28.136759 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-01ee70af-de39-414e-a3ea-501e93130154 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75626068a7784c488e8d434a95194ebb', 'request_id': 'req-01ee70af-de39-414e-a3ea-501e93130154'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:08:28.144207 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-01ee70af-de39-414e-a3ea-501e93130154 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75626068a7784c488e8d434a95194ebb', 'request_id': 'req-01ee70af-de39-414e-a3ea-501e93130154'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:08:28.157482 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-01ee70af-de39-414e-a3ea-501e93130154 admin admin] The token cache is being invalidate because project 75626068a7784c488e8d434a95194ebb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80153) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:08:28.157774 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-01ee70af-de39-414e-a3ea-501e93130154 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:08:28.158410 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-01ee70af-de39-414e-a3ea-501e93130154 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:08:28.159952 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1190/2378] 10.222.0.64 () {58 vars in 1233 bytes} [Tue Aug 2 10:08:28 2022] DELETE /identity/v3/projects/75626068a7784c488e8d434a95194ebb => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:08:28.165066 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7a5266-1fab-4540-bd18-2b00ceb3ee1e None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:08:28.165357 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7a5266-1fab-4540-bd18-2b00ceb3ee1e None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:08:28.165638 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7a5266-1fab-4540-bd18-2b00ceb3ee1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:08:28.187476 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-7e7a5266-1fab-4540-bd18-2b00ceb3ee1e None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:08:28.188729 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-7e7a5266-1fab-4540-bd18-2b00ceb3ee1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:08:28.258232 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1189/2379] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:08:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:08:28.267258 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6262b2c7-3773-4ef0-907a-91bc346d738e None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:08:28.267551 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6262b2c7-3773-4ef0-907a-91bc346d738e None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:08:28.267833 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6262b2c7-3773-4ef0-907a-91bc346d738e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:08:28.288747 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-6262b2c7-3773-4ef0-907a-91bc346d738e None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:08:28.290031 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-6262b2c7-3773-4ef0-907a-91bc346d738e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:08:28.298587 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1191/2380] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:08:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:08:28.303173 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7a5266-1fab-4540-bd18-2b00ceb3ee1e None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:08:28.314642 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7a5266-1fab-4540-bd18-2b00ceb3ee1e None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:08:28.319521 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e202615d-3985-4a9f-9cfc-69c52dbfb533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:08:28.320210 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e202615d-3985-4a9f-9cfc-69c52dbfb533 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:08:28.320686 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e202615d-3985-4a9f-9cfc-69c52dbfb533 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:08:28.321100 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e202615d-3985-4a9f-9cfc-69c52dbfb533 admin admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:08:28.321920 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e202615d-3985-4a9f-9cfc-69c52dbfb533 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:08:28.322349 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e202615d-3985-4a9f-9cfc-69c52dbfb533 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:08:28.324468 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e202615d-3985-4a9f-9cfc-69c52dbfb533 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:08:28.339191 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1190/2381] 10.222.0.64 () {58 vars in 1153 bytes} [Tue Aug 2 10:08:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:08:28.343470 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6262b2c7-3773-4ef0-907a-91bc346d738e None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:08:28.353420 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6262b2c7-3773-4ef0-907a-91bc346d738e None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:08:28.358310 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef712a4-7fe3-4e75-87b4-82861e6271bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:08:28.358934 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef712a4-7fe3-4e75-87b4-82861e6271bc admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:08:28.359220 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef712a4-7fe3-4e75-87b4-82861e6271bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:08:28.359497 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef712a4-7fe3-4e75-87b4-82861e6271bc admin admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:08:28.360181 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef712a4-7fe3-4e75-87b4-82861e6271bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:08:28.362188 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef712a4-7fe3-4e75-87b4-82861e6271bc admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:08:28.394677 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1192/2382] 10.222.0.64 () {60 vars in 1153 bytes} [Tue Aug 2 10:08:28 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:08:28.398733 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e202615d-3985-4a9f-9cfc-69c52dbfb533 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:08:28.413306 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e202615d-3985-4a9f-9cfc-69c52dbfb533 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:08:28.467826 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:08:28.489631 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f958f61-e182-4749-b3b5-3760e45a26c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:08:28.490257 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f958f61-e182-4749-b3b5-3760e45a26c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:08:28.490673 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f958f61-e182-4749-b3b5-3760e45a26c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:08:28.491167 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f958f61-e182-4749-b3b5-3760e45a26c6 admin admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:08:28.492649 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f958f61-e182-4749-b3b5-3760e45a26c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:08:28.497733 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f958f61-e182-4749-b3b5-3760e45a26c6 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:08:28.527253 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1191/2383] 10.222.0.64 () {60 vars in 1144 bytes} [Tue Aug 2 10:08:28 2022] POST /identity/v3/users => generated 287 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:08:28.534051 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef712a4-7fe3-4e75-87b4-82861e6271bc admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:08:28.545776 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef712a4-7fe3-4e75-87b4-82861e6271bc admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:08:28.551126 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dee34e-599e-490d-aca6-57129c00575d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:08:28.551808 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dee34e-599e-490d-aca6-57129c00575d admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:08:28.552183 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dee34e-599e-490d-aca6-57129c00575d admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:08:28.552494 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dee34e-599e-490d-aca6-57129c00575d admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:08:28.553264 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dee34e-599e-490d-aca6-57129c00575d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:08:28.554948 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dee34e-599e-490d-aca6-57129c00575d admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:08:28.567108 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1193/2384] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:08:28 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:08:28.571185 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f958f61-e182-4749-b3b5-3760e45a26c6 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:08:28.583048 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f958f61-e182-4749-b3b5-3760e45a26c6 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:08:28.588485 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73cdc0e-34bc-427a-9912-a9af7dd325a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:08:28.589316 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73cdc0e-34bc-427a-9912-a9af7dd325a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:08:28.589608 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73cdc0e-34bc-427a-9912-a9af7dd325a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:08:28.589893 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73cdc0e-34bc-427a-9912-a9af7dd325a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:08:28.590670 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73cdc0e-34bc-427a-9912-a9af7dd325a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:08:28.592800 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73cdc0e-34bc-427a-9912-a9af7dd325a6 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:08:28.604956 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1192/2385] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:08:28 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:08:28.610704 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dee34e-599e-490d-aca6-57129c00575d admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:08:28.624079 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dee34e-599e-490d-aca6-57129c00575d admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:08:28.628949 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d572f2-bf99-41ff-9ee8-4625d6eae593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:08:28.629739 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d572f2-bf99-41ff-9ee8-4625d6eae593 admin admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:08:28.630122 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d572f2-bf99-41ff-9ee8-4625d6eae593 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:08:28.630490 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d572f2-bf99-41ff-9ee8-4625d6eae593 admin admin] PATH_INFO: `/v3/projects/d6d54ed692f44800838863ac0d41026a/users/b241b269629a4ba6845c9475bf92ce81/roles/66c1074780954f8e87ed11a5c4e8c36a` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:08:28.653507 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d572f2-bf99-41ff-9ee8-4625d6eae593 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6d54ed692f44800838863ac0d41026a, user_id=b241b269629a4ba6845c9475bf92ce81, role_id=66c1074780954f8e87ed11a5c4e8c36a)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:08:28.656121 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d572f2-bf99-41ff-9ee8-4625d6eae593 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:08:28.671211 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1194/2386] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 10:08:28 2022] PUT /identity/v3/projects/d6d54ed692f44800838863ac0d41026a/users/b241b269629a4ba6845c9475bf92ce81/roles/66c1074780954f8e87ed11a5c4e8c36a => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:08:28.682135 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c950333f-81a9-4d8e-bb94-4e9e3735c07d None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:08:28.682675 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c950333f-81a9-4d8e-bb94-4e9e3735c07d None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:08:28.682986 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c950333f-81a9-4d8e-bb94-4e9e3735c07d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:08:28.716876 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-c950333f-81a9-4d8e-bb94-4e9e3735c07d None None] MFA Rules not processed for user `b241b269629a4ba6845c9475bf92ce81`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:08:28.719064 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-c950333f-81a9-4d8e-bb94-4e9e3735c07d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:08:28.794431 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1193/2387] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:08:28 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:08:28.799165 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d572f2-bf99-41ff-9ee8-4625d6eae593 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:08:28.821614 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:08:28.866066 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d572f2-bf99-41ff-9ee8-4625d6eae593 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:08:28.871222 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae15f41-dd51-4702-9424-d77d46f2930e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:08:28.872170 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae15f41-dd51-4702-9424-d77d46f2930e admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:08:28.872491 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae15f41-dd51-4702-9424-d77d46f2930e admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:08:28.872773 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae15f41-dd51-4702-9424-d77d46f2930e admin admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:08:28.873472 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae15f41-dd51-4702-9424-d77d46f2930e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:08:28.875399 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae15f41-dd51-4702-9424-d77d46f2930e admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:08:28.907836 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1195/2388] 10.222.0.64 () {60 vars in 1153 bytes} [Tue Aug 2 10:08:28 2022] POST /identity/v3/projects => generated 371 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:08:28.908883 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c950333f-81a9-4d8e-bb94-4e9e3735c07d None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:08:28.920413 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c950333f-81a9-4d8e-bb94-4e9e3735c07d None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:08:28.978410 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3894ba9b-5a56-4cfc-9c17-75e53580380e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:08:28.979047 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3894ba9b-5a56-4cfc-9c17-75e53580380e admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:08:28.979341 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3894ba9b-5a56-4cfc-9c17-75e53580380e admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:08:28.979622 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3894ba9b-5a56-4cfc-9c17-75e53580380e admin admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:08:28.980351 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3894ba9b-5a56-4cfc-9c17-75e53580380e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:08:28.982986 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3894ba9b-5a56-4cfc-9c17-75e53580380e admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:08:29.008307 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1194/2389] 10.222.0.64 () {60 vars in 1144 bytes} [Tue Aug 2 10:08:28 2022] POST /identity/v3/users => generated 292 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:08:29.009653 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae15f41-dd51-4702-9424-d77d46f2930e admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:08:29.019968 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae15f41-dd51-4702-9424-d77d46f2930e admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:08:29.024829 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a5e02e-2eea-4948-b135-2ddd08ef5e2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:08:29.025439 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a5e02e-2eea-4948-b135-2ddd08ef5e2c admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:08:29.025722 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a5e02e-2eea-4948-b135-2ddd08ef5e2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:08:29.025995 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a5e02e-2eea-4948-b135-2ddd08ef5e2c admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:08:29.026700 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a5e02e-2eea-4948-b135-2ddd08ef5e2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:08:29.028838 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a5e02e-2eea-4948-b135-2ddd08ef5e2c admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:08:29.039775 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1196/2390] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:08:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:08:29.044871 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3894ba9b-5a56-4cfc-9c17-75e53580380e admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:08:29.055026 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3894ba9b-5a56-4cfc-9c17-75e53580380e admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:08:29.059727 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c924c31-4193-4698-b563-7563ed209861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:08:29.060434 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c924c31-4193-4698-b563-7563ed209861 admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:08:29.060723 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c924c31-4193-4698-b563-7563ed209861 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:08:29.060996 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c924c31-4193-4698-b563-7563ed209861 admin admin] PATH_INFO: `/v3/projects/5799553a79a9443f879c6b9337665e05/users/33d9bf88f1c84a72a64d769e72b59fbc/roles/1000cf318fa14915b8091599eaead5de` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:08:29.078104 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c924c31-4193-4698-b563-7563ed209861 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5799553a79a9443f879c6b9337665e05, user_id=33d9bf88f1c84a72a64d769e72b59fbc, role_id=1000cf318fa14915b8091599eaead5de)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:08:29.080954 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c924c31-4193-4698-b563-7563ed209861 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:08:29.095744 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1195/2391] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 10:08:29 2022] PUT /identity/v3/projects/5799553a79a9443f879c6b9337665e05/users/33d9bf88f1c84a72a64d769e72b59fbc/roles/1000cf318fa14915b8091599eaead5de => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:08:29.099782 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a5e02e-2eea-4948-b135-2ddd08ef5e2c admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:08:29.111916 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a5e02e-2eea-4948-b135-2ddd08ef5e2c admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:08:29.168546 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11adcd52-3f09-4f43-96b8-766952253e74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:08:29.169680 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11adcd52-3f09-4f43-96b8-766952253e74 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:08:29.169955 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11adcd52-3f09-4f43-96b8-766952253e74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:08:29.170229 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11adcd52-3f09-4f43-96b8-766952253e74 admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:08:29.170934 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11adcd52-3f09-4f43-96b8-766952253e74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:08:29.172609 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11adcd52-3f09-4f43-96b8-766952253e74 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:08:29.183064 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1197/2392] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:08:29 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:08:29.187854 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c924c31-4193-4698-b563-7563ed209861 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:08:29.199123 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c924c31-4193-4698-b563-7563ed209861 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:08:29.204695 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1b7dbc-4e89-432c-9f63-e33d04c3ccab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:08:29.205384 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1b7dbc-4e89-432c-9f63-e33d04c3ccab admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:08:29.205671 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1b7dbc-4e89-432c-9f63-e33d04c3ccab admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:08:29.205949 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1b7dbc-4e89-432c-9f63-e33d04c3ccab admin admin] PATH_INFO: `/v3/projects/5799553a79a9443f879c6b9337665e05/users/33d9bf88f1c84a72a64d769e72b59fbc/roles/1000cf318fa14915b8091599eaead5de` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:08:29.207982 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1b7dbc-4e89-432c-9f63-e33d04c3ccab admin admin] RBAC: Authorizing `identity:create_grant(project_id=5799553a79a9443f879c6b9337665e05, user_id=33d9bf88f1c84a72a64d769e72b59fbc, role_id=1000cf318fa14915b8091599eaead5de)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:08:29.210867 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1b7dbc-4e89-432c-9f63-e33d04c3ccab admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:08:29.222638 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1196/2393] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 10:08:29 2022] PUT /identity/v3/projects/5799553a79a9443f879c6b9337665e05/users/33d9bf88f1c84a72a64d769e72b59fbc/roles/1000cf318fa14915b8091599eaead5de => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:08:29.234677 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e69fae1-0108-4ee1-a95e-68b3cde4103e None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:08:29.235017 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e69fae1-0108-4ee1-a95e-68b3cde4103e None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:08:29.235417 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e69fae1-0108-4ee1-a95e-68b3cde4103e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:08:29.264166 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-5e69fae1-0108-4ee1-a95e-68b3cde4103e None None] MFA Rules not processed for user `33d9bf88f1c84a72a64d769e72b59fbc`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:08:29.265420 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-5e69fae1-0108-4ee1-a95e-68b3cde4103e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:08:29.349533 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1198/2394] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:08:29 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:08:29.361200 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1b7dbc-4e89-432c-9f63-e33d04c3ccab admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:08:29.362882 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-4d1b7dbc-4e89-432c-9f63-e33d04c3ccab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:08:29.373646 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:08:29.422004 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1b7dbc-4e89-432c-9f63-e33d04c3ccab admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:08:29.425547 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471570c5-d229-436e-9ea0-d310c6414977 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d7de62b9ee4039bdc79627ac7fb499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:08:29.426160 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471570c5-d229-436e-9ea0-d310c6414977 service cinder] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:08:29.426441 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471570c5-d229-436e-9ea0-d310c6414977 service cinder] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:08:29.426715 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471570c5-d229-436e-9ea0-d310c6414977 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:08:29.436477 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471570c5-d229-436e-9ea0-d310c6414977 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:08:29.437911 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471570c5-d229-436e-9ea0-d310c6414977 service cinder] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:08:29.447793 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1197/2395] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:08:29 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:08:29.528437 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e69fae1-0108-4ee1-a95e-68b3cde4103e None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:08:29.538466 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e69fae1-0108-4ee1-a95e-68b3cde4103e None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:08:29.542871 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c746a4-8497-470b-9d88-69746683581d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d7de62b9ee4039bdc79627ac7fb499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:08:29.544014 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c746a4-8497-470b-9d88-69746683581d service cinder] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:08:29.544330 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c746a4-8497-470b-9d88-69746683581d service cinder] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:08:29.544687 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c746a4-8497-470b-9d88-69746683581d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:08:29.557793 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:08:29.610044 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c746a4-8497-470b-9d88-69746683581d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:08:29.611538 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c746a4-8497-470b-9d88-69746683581d service cinder] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:08:29.621046 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1199/2396] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:08:29 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:08:39.443560 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=23 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:08:39.616665 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=21 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:09:17.049266 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471570c5-d229-436e-9ea0-d310c6414977 service cinder] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:09:17.124107 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471570c5-d229-436e-9ea0-d310c6414977 service cinder] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:09:17.128793 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032a0fc5-1a84-40fd-91c6-f0aa41067ff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:09:17.129446 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032a0fc5-1a84-40fd-91c6-f0aa41067ff2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:17.129732 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032a0fc5-1a84-40fd-91c6-f0aa41067ff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:17.130009 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032a0fc5-1a84-40fd-91c6-f0aa41067ff2 admin admin] PATH_INFO: `/v3/users/b241b269629a4ba6845c9475bf92ce81` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:17.131215 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032a0fc5-1a84-40fd-91c6-f0aa41067ff2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b241b269629a4ba6845c9475bf92ce81)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:09:17.133219 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032a0fc5-1a84-40fd-91c6-f0aa41067ff2 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:09:17.189628 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-032a0fc5-1a84-40fd-91c6-f0aa41067ff2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b241b269629a4ba6845c9475bf92ce81'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:09:17.196945 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-032a0fc5-1a84-40fd-91c6-f0aa41067ff2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b241b269629a4ba6845c9475bf92ce81'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:09:17.197847 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-032a0fc5-1a84-40fd-91c6-f0aa41067ff2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b241b269629a4ba6845c9475bf92ce81'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:09:17.206282 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-032a0fc5-1a84-40fd-91c6-f0aa41067ff2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b241b269629a4ba6845c9475bf92ce81'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:09:17.223891 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-032a0fc5-1a84-40fd-91c6-f0aa41067ff2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b241b269629a4ba6845c9475bf92ce81'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:09:17.233535 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1198/2397] 10.222.0.64 () {58 vars in 1224 bytes} [Tue Aug 2 10:09:17 2022] DELETE /identity/v3/users/b241b269629a4ba6845c9475bf92ce81 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:09:17.239466 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae197dde-df5a-48fb-a0ea-a6f7d99fef57 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:17.240011 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae197dde-df5a-48fb-a0ea-a6f7d99fef57 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:17.240131 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae197dde-df5a-48fb-a0ea-a6f7d99fef57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:17.262362 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-ae197dde-df5a-48fb-a0ea-a6f7d99fef57 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:09:17.263598 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-ae197dde-df5a-48fb-a0ea-a6f7d99fef57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:09:17.343785 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1200/2398] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:09:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:09:17.355722 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032a0fc5-1a84-40fd-91c6-f0aa41067ff2 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:09:17.357827 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-032a0fc5-1a84-40fd-91c6-f0aa41067ff2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:09:17.435010 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032a0fc5-1a84-40fd-91c6-f0aa41067ff2 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:09:17.439139 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68c6b86-a1c6-4900-8dfd-8247ad885c31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:09:17.439825 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68c6b86-a1c6-4900-8dfd-8247ad885c31 service neutron] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:17.440043 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68c6b86-a1c6-4900-8dfd-8247ad885c31 service neutron] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:17.440350 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68c6b86-a1c6-4900-8dfd-8247ad885c31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:17.450813 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68c6b86-a1c6-4900-8dfd-8247ad885c31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:09:17.452576 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68c6b86-a1c6-4900-8dfd-8247ad885c31 service neutron] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:09:17.463443 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1199/2399] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:09:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:09:17.766206 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae197dde-df5a-48fb-a0ea-a6f7d99fef57 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:09:17.767849 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-ae197dde-df5a-48fb-a0ea-a6f7d99fef57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:09:17.781453 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae197dde-df5a-48fb-a0ea-a6f7d99fef57 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:09:17.787955 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eb89b7-b8e8-4eb4-97e7-4e9e6e6c627f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:09:17.789109 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eb89b7-b8e8-4eb4-97e7-4e9e6e6c627f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:17.789573 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eb89b7-b8e8-4eb4-97e7-4e9e6e6c627f admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:17.790034 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eb89b7-b8e8-4eb4-97e7-4e9e6e6c627f admin admin] PATH_INFO: `/v3/projects/d6d54ed692f44800838863ac0d41026a` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:17.796826 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41eb89b7-b8e8-4eb4-97e7-4e9e6e6c627f admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6d54ed692f44800838863ac0d41026a)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:09:17.798876 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41eb89b7-b8e8-4eb4-97e7-4e9e6e6c627f admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:09:17.857371 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-41eb89b7-b8e8-4eb4-97e7-4e9e6e6c627f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6d54ed692f44800838863ac0d41026a', 'request_id': 'req-41eb89b7-b8e8-4eb4-97e7-4e9e6e6c627f'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:09:17.863149 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-41eb89b7-b8e8-4eb4-97e7-4e9e6e6c627f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6d54ed692f44800838863ac0d41026a', 'request_id': 'req-41eb89b7-b8e8-4eb4-97e7-4e9e6e6c627f'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:09:17.869944 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-41eb89b7-b8e8-4eb4-97e7-4e9e6e6c627f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6d54ed692f44800838863ac0d41026a', 'request_id': 'req-41eb89b7-b8e8-4eb4-97e7-4e9e6e6c627f'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:09:17.883715 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-41eb89b7-b8e8-4eb4-97e7-4e9e6e6c627f admin admin] The token cache is being invalidate because project d6d54ed692f44800838863ac0d41026a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80153) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:09:17.884039 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-41eb89b7-b8e8-4eb4-97e7-4e9e6e6c627f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:09:17.884778 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-41eb89b7-b8e8-4eb4-97e7-4e9e6e6c627f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:09:17.886563 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1201/2400] 10.222.0.64 () {58 vars in 1233 bytes} [Tue Aug 2 10:09:17 2022] DELETE /identity/v3/projects/d6d54ed692f44800838863ac0d41026a => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:09:17.890865 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68c6b86-a1c6-4900-8dfd-8247ad885c31 service neutron] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:09:17.902283 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68c6b86-a1c6-4900-8dfd-8247ad885c31 service neutron] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:09:17.923270 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:09:17.980597 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465447e5-1455-404b-b3d0-e97ced65973e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:09:17.981149 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465447e5-1455-404b-b3d0-e97ced65973e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:17.981433 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465447e5-1455-404b-b3d0-e97ced65973e admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:17.981709 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465447e5-1455-404b-b3d0-e97ced65973e admin admin] PATH_INFO: `/v3/users/33d9bf88f1c84a72a64d769e72b59fbc` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:17.982918 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465447e5-1455-404b-b3d0-e97ced65973e admin admin] RBAC: Authorizing `identity:delete_user(user_id=33d9bf88f1c84a72a64d769e72b59fbc)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:09:17.984906 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465447e5-1455-404b-b3d0-e97ced65973e admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:09:18.040150 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-465447e5-1455-404b-b3d0-e97ced65973e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33d9bf88f1c84a72a64d769e72b59fbc'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:09:18.047647 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-465447e5-1455-404b-b3d0-e97ced65973e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33d9bf88f1c84a72a64d769e72b59fbc'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:09:18.048481 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-465447e5-1455-404b-b3d0-e97ced65973e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33d9bf88f1c84a72a64d769e72b59fbc'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:09:18.058584 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-465447e5-1455-404b-b3d0-e97ced65973e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33d9bf88f1c84a72a64d769e72b59fbc'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:09:18.074523 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-465447e5-1455-404b-b3d0-e97ced65973e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33d9bf88f1c84a72a64d769e72b59fbc'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:09:18.080713 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1200/2401] 10.222.0.64 () {58 vars in 1224 bytes} [Tue Aug 2 10:09:17 2022] DELETE /identity/v3/users/33d9bf88f1c84a72a64d769e72b59fbc => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:09:18.381984 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eb89b7-b8e8-4eb4-97e7-4e9e6e6c627f admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:09:18.383549 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-41eb89b7-b8e8-4eb4-97e7-4e9e6e6c627f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:09:18.407748 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eb89b7-b8e8-4eb4-97e7-4e9e6e6c627f admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:09:18.476068 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ff7afd-488d-4031-ad48-4988084d2c01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:09:18.476862 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ff7afd-488d-4031-ad48-4988084d2c01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:18.477217 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ff7afd-488d-4031-ad48-4988084d2c01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:18.477571 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ff7afd-488d-4031-ad48-4988084d2c01 admin admin] PATH_INFO: `/v3/projects/5799553a79a9443f879c6b9337665e05` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:18.478759 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ff7afd-488d-4031-ad48-4988084d2c01 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5799553a79a9443f879c6b9337665e05)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:09:18.480796 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ff7afd-488d-4031-ad48-4988084d2c01 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:09:18.526439 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:09:18.550741 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-f6ff7afd-488d-4031-ad48-4988084d2c01 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5799553a79a9443f879c6b9337665e05', 'request_id': 'req-f6ff7afd-488d-4031-ad48-4988084d2c01'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:09:18.559450 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-f6ff7afd-488d-4031-ad48-4988084d2c01 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5799553a79a9443f879c6b9337665e05', 'request_id': 'req-f6ff7afd-488d-4031-ad48-4988084d2c01'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:09:18.567530 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-f6ff7afd-488d-4031-ad48-4988084d2c01 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5799553a79a9443f879c6b9337665e05', 'request_id': 'req-f6ff7afd-488d-4031-ad48-4988084d2c01'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:09:18.584118 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-f6ff7afd-488d-4031-ad48-4988084d2c01 admin admin] The token cache is being invalidate because project 5799553a79a9443f879c6b9337665e05 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80153) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:09:18.584452 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-f6ff7afd-488d-4031-ad48-4988084d2c01 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:09:18.585141 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-f6ff7afd-488d-4031-ad48-4988084d2c01 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:09:18.586802 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1202/2402] 10.222.0.64 () {58 vars in 1233 bytes} [Tue Aug 2 10:09:18 2022] DELETE /identity/v3/projects/5799553a79a9443f879c6b9337665e05 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:09:18.592148 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5e81cd-d965-49fc-9408-baa7ce8e4f79 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:18.592657 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5e81cd-d965-49fc-9408-baa7ce8e4f79 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:18.592959 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5e81cd-d965-49fc-9408-baa7ce8e4f79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:18.615329 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-5f5e81cd-d965-49fc-9408-baa7ce8e4f79 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:09:18.616803 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-5f5e81cd-d965-49fc-9408-baa7ce8e4f79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:09:18.707951 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1201/2403] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:09:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:09:18.716462 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1976e18-20f4-4899-8812-18ee07c67213 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:18.716748 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1976e18-20f4-4899-8812-18ee07c67213 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:18.717029 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1976e18-20f4-4899-8812-18ee07c67213 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:18.746547 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-d1976e18-20f4-4899-8812-18ee07c67213 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:09:18.748519 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-d1976e18-20f4-4899-8812-18ee07c67213 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:09:18.756857 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1203/2404] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:09:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:09:18.765321 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5e81cd-d965-49fc-9408-baa7ce8e4f79 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:09:18.777158 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5e81cd-d965-49fc-9408-baa7ce8e4f79 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:09:18.783069 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8916df-d6df-4d9f-b632-d4d98b37cfe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:09:18.783794 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8916df-d6df-4d9f-b632-d4d98b37cfe9 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:18.784148 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8916df-d6df-4d9f-b632-d4d98b37cfe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:18.784530 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8916df-d6df-4d9f-b632-d4d98b37cfe9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:18.785336 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8916df-d6df-4d9f-b632-d4d98b37cfe9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:09:18.785652 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8916df-d6df-4d9f-b632-d4d98b37cfe9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:09:18.787718 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8916df-d6df-4d9f-b632-d4d98b37cfe9 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:09:18.806393 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1202/2405] 10.222.0.64 () {58 vars in 1153 bytes} [Tue Aug 2 10:09:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:09:18.812117 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1976e18-20f4-4899-8812-18ee07c67213 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:09:18.824808 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1976e18-20f4-4899-8812-18ee07c67213 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:09:18.830417 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d97f58-0a86-4aa0-878c-c75fd01ac212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:09:18.831191 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d97f58-0a86-4aa0-878c-c75fd01ac212 admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:18.831487 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d97f58-0a86-4aa0-878c-c75fd01ac212 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:18.831876 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d97f58-0a86-4aa0-878c-c75fd01ac212 admin admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:18.832532 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d97f58-0a86-4aa0-878c-c75fd01ac212 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:09:18.834481 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d97f58-0a86-4aa0-878c-c75fd01ac212 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:09:18.871759 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1204/2406] 10.222.0.64 () {60 vars in 1153 bytes} [Tue Aug 2 10:09:18 2022] POST /identity/v3/projects => generated 363 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:09:18.878303 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8916df-d6df-4d9f-b632-d4d98b37cfe9 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:09:18.891089 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8916df-d6df-4d9f-b632-d4d98b37cfe9 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:09:18.927243 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:09:18.966292 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b293599-f9f0-4d33-81ed-5268297258e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:09:18.966931 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b293599-f9f0-4d33-81ed-5268297258e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:18.967217 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b293599-f9f0-4d33-81ed-5268297258e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:18.967489 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b293599-f9f0-4d33-81ed-5268297258e5 admin admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:18.968173 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b293599-f9f0-4d33-81ed-5268297258e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:09:18.970149 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b293599-f9f0-4d33-81ed-5268297258e5 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:09:18.993932 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1203/2407] 10.222.0.64 () {60 vars in 1144 bytes} [Tue Aug 2 10:09:18 2022] POST /identity/v3/users => generated 282 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:09:18.998183 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d97f58-0a86-4aa0-878c-c75fd01ac212 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:09:19.010032 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d97f58-0a86-4aa0-878c-c75fd01ac212 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:09:19.015263 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9e676b-beda-4914-a59c-ccedc75be07f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:09:19.015883 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9e676b-beda-4914-a59c-ccedc75be07f admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:19.016165 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9e676b-beda-4914-a59c-ccedc75be07f admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:19.016480 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9e676b-beda-4914-a59c-ccedc75be07f admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:19.017189 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9e676b-beda-4914-a59c-ccedc75be07f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:09:19.018789 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9e676b-beda-4914-a59c-ccedc75be07f admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:09:19.032759 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1205/2408] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:09:18 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:09:19.037497 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b293599-f9f0-4d33-81ed-5268297258e5 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:09:19.048080 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b293599-f9f0-4d33-81ed-5268297258e5 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:09:19.054575 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4b3f65-e37c-4dc3-8a91-78486cd2537a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:09:19.055200 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4b3f65-e37c-4dc3-8a91-78486cd2537a admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:19.055516 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4b3f65-e37c-4dc3-8a91-78486cd2537a admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:19.055791 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4b3f65-e37c-4dc3-8a91-78486cd2537a admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:19.057000 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4b3f65-e37c-4dc3-8a91-78486cd2537a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:09:19.059453 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4b3f65-e37c-4dc3-8a91-78486cd2537a admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:09:19.071933 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1204/2409] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:09:19 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:09:19.076960 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9e676b-beda-4914-a59c-ccedc75be07f admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:09:19.087410 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9e676b-beda-4914-a59c-ccedc75be07f admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:09:19.092319 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa6ef8f-2461-4fcb-9800-eb66b9beec78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:09:19.092994 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa6ef8f-2461-4fcb-9800-eb66b9beec78 admin admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:19.093277 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa6ef8f-2461-4fcb-9800-eb66b9beec78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:19.093556 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa6ef8f-2461-4fcb-9800-eb66b9beec78 admin admin] PATH_INFO: `/v3/projects/8369dc1a4e264cc9905b1630554a5dd4/users/c6afc30695c94832a7ab6a58e72b83d8/roles/66c1074780954f8e87ed11a5c4e8c36a` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:19.113426 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa6ef8f-2461-4fcb-9800-eb66b9beec78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8369dc1a4e264cc9905b1630554a5dd4, user_id=c6afc30695c94832a7ab6a58e72b83d8, role_id=66c1074780954f8e87ed11a5c4e8c36a)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:09:19.116526 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa6ef8f-2461-4fcb-9800-eb66b9beec78 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:09:19.131941 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1206/2410] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 10:09:19 2022] PUT /identity/v3/projects/8369dc1a4e264cc9905b1630554a5dd4/users/c6afc30695c94832a7ab6a58e72b83d8/roles/66c1074780954f8e87ed11a5c4e8c36a => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:09:19.143838 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44edf7e7-c8ea-4ecc-8166-eb3a54013c1b None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:19.144130 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44edf7e7-c8ea-4ecc-8166-eb3a54013c1b None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:19.144597 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44edf7e7-c8ea-4ecc-8166-eb3a54013c1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:19.179140 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-44edf7e7-c8ea-4ecc-8166-eb3a54013c1b None None] MFA Rules not processed for user `c6afc30695c94832a7ab6a58e72b83d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:09:19.180405 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-44edf7e7-c8ea-4ecc-8166-eb3a54013c1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:09:19.260003 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1205/2411] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:09:19 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:09:19.267948 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa6ef8f-2461-4fcb-9800-eb66b9beec78 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:09:19.343470 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa6ef8f-2461-4fcb-9800-eb66b9beec78 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:09:19.353818 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb92dd1-5c1b-4ff9-85f3-6879c8e77e07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:09:19.354494 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb92dd1-5c1b-4ff9-85f3-6879c8e77e07 admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:19.354779 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb92dd1-5c1b-4ff9-85f3-6879c8e77e07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:19.355057 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb92dd1-5c1b-4ff9-85f3-6879c8e77e07 admin admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:19.355739 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb92dd1-5c1b-4ff9-85f3-6879c8e77e07 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:09:19.357720 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb92dd1-5c1b-4ff9-85f3-6879c8e77e07 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:09:19.392937 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1207/2412] 10.222.0.64 () {60 vars in 1153 bytes} [Tue Aug 2 10:09:19 2022] POST /identity/v3/projects => generated 365 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:09:19.397376 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44edf7e7-c8ea-4ecc-8166-eb3a54013c1b None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:09:19.409741 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44edf7e7-c8ea-4ecc-8166-eb3a54013c1b None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:09:19.473959 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b84320b-8b5c-4098-835e-f4fd2f633df8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:09:19.474618 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b84320b-8b5c-4098-835e-f4fd2f633df8 admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:19.474906 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b84320b-8b5c-4098-835e-f4fd2f633df8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:19.475183 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b84320b-8b5c-4098-835e-f4fd2f633df8 admin admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:19.475895 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b84320b-8b5c-4098-835e-f4fd2f633df8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:09:19.477952 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b84320b-8b5c-4098-835e-f4fd2f633df8 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:09:19.502750 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1206/2413] 10.222.0.64 () {60 vars in 1144 bytes} [Tue Aug 2 10:09:19 2022] POST /identity/v3/users => generated 289 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:09:19.507395 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb92dd1-5c1b-4ff9-85f3-6879c8e77e07 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:09:19.518588 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb92dd1-5c1b-4ff9-85f3-6879c8e77e07 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:09:19.523649 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafc7805-3024-47a2-b06a-a400a8477f49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:09:19.524326 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafc7805-3024-47a2-b06a-a400a8477f49 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:19.524611 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafc7805-3024-47a2-b06a-a400a8477f49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:19.524884 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafc7805-3024-47a2-b06a-a400a8477f49 admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:19.525598 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafc7805-3024-47a2-b06a-a400a8477f49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:09:19.527236 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafc7805-3024-47a2-b06a-a400a8477f49 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:09:19.539445 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1208/2414] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:09:19 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:09:19.544470 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b84320b-8b5c-4098-835e-f4fd2f633df8 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:09:19.555303 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b84320b-8b5c-4098-835e-f4fd2f633df8 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:09:19.561075 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ef3ddd-545c-4730-bd5e-23a8a6789c94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:09:19.561780 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ef3ddd-545c-4730-bd5e-23a8a6789c94 admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:19.562067 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ef3ddd-545c-4730-bd5e-23a8a6789c94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:19.562349 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ef3ddd-545c-4730-bd5e-23a8a6789c94 admin admin] PATH_INFO: `/v3/projects/5e20ef9838864bc0814705461cd0adec/users/c5d4405194154ab19f1af8a255bd14f2/roles/1000cf318fa14915b8091599eaead5de` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:19.582246 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ef3ddd-545c-4730-bd5e-23a8a6789c94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e20ef9838864bc0814705461cd0adec, user_id=c5d4405194154ab19f1af8a255bd14f2, role_id=1000cf318fa14915b8091599eaead5de)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:09:19.585108 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ef3ddd-545c-4730-bd5e-23a8a6789c94 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:09:19.602332 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1207/2415] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 10:09:19 2022] PUT /identity/v3/projects/5e20ef9838864bc0814705461cd0adec/users/c5d4405194154ab19f1af8a255bd14f2/roles/1000cf318fa14915b8091599eaead5de => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:09:19.607253 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafc7805-3024-47a2-b06a-a400a8477f49 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:09:19.617928 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafc7805-3024-47a2-b06a-a400a8477f49 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:09:19.640022 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:09:19.684105 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c644190d-4b17-4999-ab25-4245ad4d606e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:09:19.684758 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c644190d-4b17-4999-ab25-4245ad4d606e admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:19.685044 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c644190d-4b17-4999-ab25-4245ad4d606e admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:19.685321 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c644190d-4b17-4999-ab25-4245ad4d606e admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:19.686022 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c644190d-4b17-4999-ab25-4245ad4d606e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:09:19.687630 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c644190d-4b17-4999-ab25-4245ad4d606e admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:09:19.699069 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1209/2416] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:09:19 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:09:19.703294 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ef3ddd-545c-4730-bd5e-23a8a6789c94 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:09:19.709002 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:09:19.716029 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ef3ddd-545c-4730-bd5e-23a8a6789c94 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:09:19.722313 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bde0eb2-1f11-4756-b4b8-e76daaff7a86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:09:19.723349 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bde0eb2-1f11-4756-b4b8-e76daaff7a86 admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:19.723970 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bde0eb2-1f11-4756-b4b8-e76daaff7a86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:19.724158 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bde0eb2-1f11-4756-b4b8-e76daaff7a86 admin admin] PATH_INFO: `/v3/projects/5e20ef9838864bc0814705461cd0adec/users/c5d4405194154ab19f1af8a255bd14f2/roles/1000cf318fa14915b8091599eaead5de` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:19.726350 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bde0eb2-1f11-4756-b4b8-e76daaff7a86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e20ef9838864bc0814705461cd0adec, user_id=c5d4405194154ab19f1af8a255bd14f2, role_id=1000cf318fa14915b8091599eaead5de)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:09:19.729099 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bde0eb2-1f11-4756-b4b8-e76daaff7a86 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:09:19.746436 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1208/2417] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 10:09:19 2022] PUT /identity/v3/projects/5e20ef9838864bc0814705461cd0adec/users/c5d4405194154ab19f1af8a255bd14f2/roles/1000cf318fa14915b8091599eaead5de => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:09:19.757737 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f56ded-d0e2-4e00-865b-3af510569dd0 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:19.758139 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f56ded-d0e2-4e00-865b-3af510569dd0 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:19.758364 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f56ded-d0e2-4e00-865b-3af510569dd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:19.799625 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-d6f56ded-d0e2-4e00-865b-3af510569dd0 None None] MFA Rules not processed for user `c5d4405194154ab19f1af8a255bd14f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:09:19.801576 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-d6f56ded-d0e2-4e00-865b-3af510569dd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:09:19.898387 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1210/2418] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:09:19 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:09:19.910663 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bde0eb2-1f11-4756-b4b8-e76daaff7a86 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:09:19.912868 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-7bde0eb2-1f11-4756-b4b8-e76daaff7a86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:09:19.991971 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bde0eb2-1f11-4756-b4b8-e76daaff7a86 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:09:19.995836 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13183308-6365-4eeb-adf5-5e78ca88d38f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d7de62b9ee4039bdc79627ac7fb499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:09:19.996502 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13183308-6365-4eeb-adf5-5e78ca88d38f service cinder] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:19.996835 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13183308-6365-4eeb-adf5-5e78ca88d38f service cinder] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:19.997117 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13183308-6365-4eeb-adf5-5e78ca88d38f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:20.007076 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13183308-6365-4eeb-adf5-5e78ca88d38f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:09:20.008564 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13183308-6365-4eeb-adf5-5e78ca88d38f service cinder] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:09:20.020181 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1209/2419] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:09:19 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:09:20.495794 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f56ded-d0e2-4e00-865b-3af510569dd0 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:09:20.566085 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f56ded-d0e2-4e00-865b-3af510569dd0 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:09:20.570607 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72a0986-1829-433e-801b-24d4c39fa58e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:09:20.571246 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72a0986-1829-433e-801b-24d4c39fa58e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:20.571528 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72a0986-1829-433e-801b-24d4c39fa58e admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:20.571802 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72a0986-1829-433e-801b-24d4c39fa58e admin admin] PATH_INFO: `/v3/users/c6afc30695c94832a7ab6a58e72b83d8` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:20.573589 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72a0986-1829-433e-801b-24d4c39fa58e admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6afc30695c94832a7ab6a58e72b83d8)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:09:20.575560 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72a0986-1829-433e-801b-24d4c39fa58e admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:09:20.625489 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-a72a0986-1829-433e-801b-24d4c39fa58e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6afc30695c94832a7ab6a58e72b83d8'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:09:20.626696 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:09:20.634749 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-a72a0986-1829-433e-801b-24d4c39fa58e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6afc30695c94832a7ab6a58e72b83d8'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:09:20.643246 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-a72a0986-1829-433e-801b-24d4c39fa58e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6afc30695c94832a7ab6a58e72b83d8'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:09:20.643887 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-a72a0986-1829-433e-801b-24d4c39fa58e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6afc30695c94832a7ab6a58e72b83d8'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:09:20.644537 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-a72a0986-1829-433e-801b-24d4c39fa58e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6afc30695c94832a7ab6a58e72b83d8'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:09:20.662293 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1211/2420] 10.222.0.64 () {58 vars in 1224 bytes} [Tue Aug 2 10:09:20 2022] DELETE /identity/v3/users/c6afc30695c94832a7ab6a58e72b83d8 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:09:20.667210 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b4279d-b0dc-4dfe-b514-841a6504b0bd None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:20.667789 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b4279d-b0dc-4dfe-b514-841a6504b0bd None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:20.667789 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b4279d-b0dc-4dfe-b514-841a6504b0bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:20.691570 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-f1b4279d-b0dc-4dfe-b514-841a6504b0bd None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:09:20.692856 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-f1b4279d-b0dc-4dfe-b514-841a6504b0bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:09:20.758300 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1210/2421] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:09:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:09:20.769946 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72a0986-1829-433e-801b-24d4c39fa58e admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:09:20.771466 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-a72a0986-1829-433e-801b-24d4c39fa58e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:09:20.837110 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72a0986-1829-433e-801b-24d4c39fa58e admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:09:20.841166 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2074e79-59a1-4edc-976f-ac6751684960 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:09:20.841802 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2074e79-59a1-4edc-976f-ac6751684960 service neutron] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:20.842084 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2074e79-59a1-4edc-976f-ac6751684960 service neutron] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:20.842364 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2074e79-59a1-4edc-976f-ac6751684960 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:20.852576 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2074e79-59a1-4edc-976f-ac6751684960 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:09:20.854053 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2074e79-59a1-4edc-976f-ac6751684960 service neutron] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:09:20.864340 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1212/2422] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:09:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:09:21.153272 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b4279d-b0dc-4dfe-b514-841a6504b0bd None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:09:21.154945 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-f1b4279d-b0dc-4dfe-b514-841a6504b0bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:09:21.183365 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b4279d-b0dc-4dfe-b514-841a6504b0bd None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:09:21.188742 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a838ee34-6b5f-473d-910e-5d731938d3bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:09:21.189434 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a838ee34-6b5f-473d-910e-5d731938d3bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:21.189686 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a838ee34-6b5f-473d-910e-5d731938d3bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:21.189957 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a838ee34-6b5f-473d-910e-5d731938d3bb admin admin] PATH_INFO: `/v3/projects/8369dc1a4e264cc9905b1630554a5dd4` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:21.191162 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a838ee34-6b5f-473d-910e-5d731938d3bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=8369dc1a4e264cc9905b1630554a5dd4)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:09:21.193194 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a838ee34-6b5f-473d-910e-5d731938d3bb admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:09:21.249118 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-a838ee34-6b5f-473d-910e-5d731938d3bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8369dc1a4e264cc9905b1630554a5dd4', 'request_id': 'req-a838ee34-6b5f-473d-910e-5d731938d3bb'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:09:21.255400 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-a838ee34-6b5f-473d-910e-5d731938d3bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8369dc1a4e264cc9905b1630554a5dd4', 'request_id': 'req-a838ee34-6b5f-473d-910e-5d731938d3bb'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:09:21.262024 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-a838ee34-6b5f-473d-910e-5d731938d3bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8369dc1a4e264cc9905b1630554a5dd4', 'request_id': 'req-a838ee34-6b5f-473d-910e-5d731938d3bb'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:09:21.275279 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-a838ee34-6b5f-473d-910e-5d731938d3bb admin admin] The token cache is being invalidate because project 8369dc1a4e264cc9905b1630554a5dd4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80154) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:09:21.275579 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-a838ee34-6b5f-473d-910e-5d731938d3bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:09:21.276222 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-a838ee34-6b5f-473d-910e-5d731938d3bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:09:21.277850 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1211/2423] 10.222.0.64 () {58 vars in 1233 bytes} [Tue Aug 2 10:09:21 2022] DELETE /identity/v3/projects/8369dc1a4e264cc9905b1630554a5dd4 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:09:21.281914 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2074e79-59a1-4edc-976f-ac6751684960 service neutron] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:09:21.292153 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2074e79-59a1-4edc-976f-ac6751684960 service neutron] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:09:21.355292 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815ae832-c587-43ed-84f3-d665c92e77fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:09:21.355863 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815ae832-c587-43ed-84f3-d665c92e77fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:21.356155 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815ae832-c587-43ed-84f3-d665c92e77fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:21.356464 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815ae832-c587-43ed-84f3-d665c92e77fc admin admin] PATH_INFO: `/v3/users/c5d4405194154ab19f1af8a255bd14f2` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:21.357648 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815ae832-c587-43ed-84f3-d665c92e77fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5d4405194154ab19f1af8a255bd14f2)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:09:21.359598 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815ae832-c587-43ed-84f3-d665c92e77fc admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:09:21.411575 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-815ae832-c587-43ed-84f3-d665c92e77fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5d4405194154ab19f1af8a255bd14f2'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:09:21.420874 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-815ae832-c587-43ed-84f3-d665c92e77fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5d4405194154ab19f1af8a255bd14f2'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:09:21.430290 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-815ae832-c587-43ed-84f3-d665c92e77fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5d4405194154ab19f1af8a255bd14f2'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:09:21.431062 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-815ae832-c587-43ed-84f3-d665c92e77fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5d4405194154ab19f1af8a255bd14f2'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:09:21.432374 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-815ae832-c587-43ed-84f3-d665c92e77fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5d4405194154ab19f1af8a255bd14f2'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:09:21.441729 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:09:21.462832 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1213/2424] 10.222.0.64 () {58 vars in 1224 bytes} [Tue Aug 2 10:09:21 2022] DELETE /identity/v3/users/c5d4405194154ab19f1af8a255bd14f2 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:09:21.771785 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a838ee34-6b5f-473d-910e-5d731938d3bb admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:09:21.773773 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-a838ee34-6b5f-473d-910e-5d731938d3bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:09:21.794565 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a838ee34-6b5f-473d-910e-5d731938d3bb admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:09:21.810993 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:09:21.878232 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ba929c-87c5-4090-8dfb-e3949b258972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:09:21.878846 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ba929c-87c5-4090-8dfb-e3949b258972 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:21.879239 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ba929c-87c5-4090-8dfb-e3949b258972 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:21.879631 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ba929c-87c5-4090-8dfb-e3949b258972 admin admin] PATH_INFO: `/v3/projects/5e20ef9838864bc0814705461cd0adec` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:21.881086 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ba929c-87c5-4090-8dfb-e3949b258972 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e20ef9838864bc0814705461cd0adec)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:09:21.883309 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ba929c-87c5-4090-8dfb-e3949b258972 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:09:21.941700 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-32ba929c-87c5-4090-8dfb-e3949b258972 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e20ef9838864bc0814705461cd0adec', 'request_id': 'req-32ba929c-87c5-4090-8dfb-e3949b258972'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:09:21.947587 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-32ba929c-87c5-4090-8dfb-e3949b258972 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e20ef9838864bc0814705461cd0adec', 'request_id': 'req-32ba929c-87c5-4090-8dfb-e3949b258972'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:09:21.954929 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-32ba929c-87c5-4090-8dfb-e3949b258972 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e20ef9838864bc0814705461cd0adec', 'request_id': 'req-32ba929c-87c5-4090-8dfb-e3949b258972'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:09:21.971955 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-32ba929c-87c5-4090-8dfb-e3949b258972 admin admin] The token cache is being invalidate because project 5e20ef9838864bc0814705461cd0adec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80154) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:09:21.971955 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-32ba929c-87c5-4090-8dfb-e3949b258972 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:09:21.972905 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-32ba929c-87c5-4090-8dfb-e3949b258972 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:09:21.974471 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1212/2425] 10.222.0.64 () {58 vars in 1233 bytes} [Tue Aug 2 10:09:21 2022] DELETE /identity/v3/projects/5e20ef9838864bc0814705461cd0adec => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:09:21.980747 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1817bda3-5dd3-45bb-b667-9b2c2f1f547d None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:21.981532 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1817bda3-5dd3-45bb-b667-9b2c2f1f547d None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:21.981744 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1817bda3-5dd3-45bb-b667-9b2c2f1f547d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:22.003794 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-1817bda3-5dd3-45bb-b667-9b2c2f1f547d None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:09:22.005147 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-1817bda3-5dd3-45bb-b667-9b2c2f1f547d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:09:22.078067 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1214/2426] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:09:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:09:22.085995 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cfce4b-c417-4011-8dff-faa48a902c2c None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:22.086504 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cfce4b-c417-4011-8dff-faa48a902c2c None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:22.086853 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cfce4b-c417-4011-8dff-faa48a902c2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:22.106570 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-89cfce4b-c417-4011-8dff-faa48a902c2c None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:09:22.107777 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-89cfce4b-c417-4011-8dff-faa48a902c2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:09:22.116175 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1213/2427] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:09:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:09:22.121145 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1817bda3-5dd3-45bb-b667-9b2c2f1f547d None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:09:22.131039 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1817bda3-5dd3-45bb-b667-9b2c2f1f547d None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:09:22.136238 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1404ed21-1597-47c1-b94a-8dc97e96078e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:09:22.136884 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1404ed21-1597-47c1-b94a-8dc97e96078e admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:22.137171 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1404ed21-1597-47c1-b94a-8dc97e96078e admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:22.137445 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1404ed21-1597-47c1-b94a-8dc97e96078e admin admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:22.138151 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1404ed21-1597-47c1-b94a-8dc97e96078e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:09:22.138456 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1404ed21-1597-47c1-b94a-8dc97e96078e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:09:22.140066 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1404ed21-1597-47c1-b94a-8dc97e96078e admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:09:22.152576 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1215/2428] 10.222.0.64 () {58 vars in 1153 bytes} [Tue Aug 2 10:09:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:09:22.157173 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cfce4b-c417-4011-8dff-faa48a902c2c None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:09:22.166786 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cfce4b-c417-4011-8dff-faa48a902c2c None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:09:22.171584 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45f18ff-b871-4d59-92f5-a33392efdbee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:09:22.172200 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45f18ff-b871-4d59-92f5-a33392efdbee admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:22.172510 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45f18ff-b871-4d59-92f5-a33392efdbee admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:22.172784 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45f18ff-b871-4d59-92f5-a33392efdbee admin admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:22.173454 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45f18ff-b871-4d59-92f5-a33392efdbee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:09:22.175422 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45f18ff-b871-4d59-92f5-a33392efdbee admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:09:22.204734 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1214/2429] 10.222.0.64 () {60 vars in 1153 bytes} [Tue Aug 2 10:09:22 2022] POST /identity/v3/projects => generated 363 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:09:22.209439 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1404ed21-1597-47c1-b94a-8dc97e96078e admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:09:22.219730 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1404ed21-1597-47c1-b94a-8dc97e96078e admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:09:22.285257 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60801338-5fd5-47bb-ba58-687030abf5fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:09:22.285883 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60801338-5fd5-47bb-ba58-687030abf5fa admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:22.286171 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60801338-5fd5-47bb-ba58-687030abf5fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:22.286451 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60801338-5fd5-47bb-ba58-687030abf5fa admin admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:22.287133 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60801338-5fd5-47bb-ba58-687030abf5fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:09:22.289239 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60801338-5fd5-47bb-ba58-687030abf5fa admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:09:22.311060 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1216/2430] 10.222.0.64 () {60 vars in 1144 bytes} [Tue Aug 2 10:09:22 2022] POST /identity/v3/users => generated 282 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:09:22.315822 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45f18ff-b871-4d59-92f5-a33392efdbee admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:09:22.327318 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45f18ff-b871-4d59-92f5-a33392efdbee admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:09:22.338076 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8adec10-9aa5-4975-a077-33180bdc3fa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:09:22.338715 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8adec10-9aa5-4975-a077-33180bdc3fa6 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:22.339002 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8adec10-9aa5-4975-a077-33180bdc3fa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:22.339279 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8adec10-9aa5-4975-a077-33180bdc3fa6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:22.340018 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8adec10-9aa5-4975-a077-33180bdc3fa6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:09:22.341814 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8adec10-9aa5-4975-a077-33180bdc3fa6 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:09:22.352172 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1215/2431] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:09:22 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:09:22.356306 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60801338-5fd5-47bb-ba58-687030abf5fa admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:09:22.366408 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60801338-5fd5-47bb-ba58-687030abf5fa admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:09:22.371892 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddcc541-4e3c-48ee-bca1-efc1477a86a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:09:22.372538 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddcc541-4e3c-48ee-bca1-efc1477a86a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:22.372829 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddcc541-4e3c-48ee-bca1-efc1477a86a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:22.373109 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddcc541-4e3c-48ee-bca1-efc1477a86a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:22.373813 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddcc541-4e3c-48ee-bca1-efc1477a86a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:09:22.375421 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddcc541-4e3c-48ee-bca1-efc1477a86a4 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:09:22.387078 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1217/2432] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:09:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:09:22.391216 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8adec10-9aa5-4975-a077-33180bdc3fa6 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:09:22.401759 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8adec10-9aa5-4975-a077-33180bdc3fa6 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:09:22.406453 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eb445b-6881-433a-849b-b57c5d56865e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:09:22.407112 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eb445b-6881-433a-849b-b57c5d56865e admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:22.407393 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eb445b-6881-433a-849b-b57c5d56865e admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:22.407670 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eb445b-6881-433a-849b-b57c5d56865e admin admin] PATH_INFO: `/v3/projects/7ac1d1647c844d6e8d8037ca1cd4431f/users/ef4d2afb79b94ca58010725e332e44d7/roles/66c1074780954f8e87ed11a5c4e8c36a` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:22.422128 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00eb445b-6881-433a-849b-b57c5d56865e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ac1d1647c844d6e8d8037ca1cd4431f, user_id=ef4d2afb79b94ca58010725e332e44d7, role_id=66c1074780954f8e87ed11a5c4e8c36a)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:09:22.424795 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00eb445b-6881-433a-849b-b57c5d56865e admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:09:22.443431 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1216/2433] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 10:09:22 2022] PUT /identity/v3/projects/7ac1d1647c844d6e8d8037ca1cd4431f/users/ef4d2afb79b94ca58010725e332e44d7/roles/66c1074780954f8e87ed11a5c4e8c36a => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:09:22.517035 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112aadbd-96ff-439b-af6a-96cfce7bdd2a None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:22.517324 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112aadbd-96ff-439b-af6a-96cfce7bdd2a None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:22.517604 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112aadbd-96ff-439b-af6a-96cfce7bdd2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:22.548262 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-112aadbd-96ff-439b-af6a-96cfce7bdd2a None None] MFA Rules not processed for user `ef4d2afb79b94ca58010725e332e44d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:09:22.549495 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-112aadbd-96ff-439b-af6a-96cfce7bdd2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:09:22.593318 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:09:22.621563 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1218/2434] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:09:22 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:09:22.626122 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eb445b-6881-433a-849b-b57c5d56865e admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:09:22.691086 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:09:22.711871 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eb445b-6881-433a-849b-b57c5d56865e admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:09:22.716624 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045336fb-a26e-4c00-9c92-e1fdc9ac593f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:09:22.717230 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045336fb-a26e-4c00-9c92-e1fdc9ac593f admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:22.717513 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045336fb-a26e-4c00-9c92-e1fdc9ac593f admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:22.717786 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045336fb-a26e-4c00-9c92-e1fdc9ac593f admin admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:22.718453 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045336fb-a26e-4c00-9c92-e1fdc9ac593f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:09:22.720388 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045336fb-a26e-4c00-9c92-e1fdc9ac593f admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:09:22.749765 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1217/2435] 10.222.0.64 () {60 vars in 1153 bytes} [Tue Aug 2 10:09:22 2022] POST /identity/v3/projects => generated 363 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:09:22.753883 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112aadbd-96ff-439b-af6a-96cfce7bdd2a None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:09:22.764817 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112aadbd-96ff-439b-af6a-96cfce7bdd2a None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:09:22.829304 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad7c2de-c293-4c4c-a2fc-a6db0c294424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:09:22.829945 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad7c2de-c293-4c4c-a2fc-a6db0c294424 admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:22.830266 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad7c2de-c293-4c4c-a2fc-a6db0c294424 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:22.830546 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad7c2de-c293-4c4c-a2fc-a6db0c294424 admin admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:22.831230 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad7c2de-c293-4c4c-a2fc-a6db0c294424 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:09:22.833275 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad7c2de-c293-4c4c-a2fc-a6db0c294424 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:09:22.864364 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1219/2436] 10.222.0.64 () {60 vars in 1144 bytes} [Tue Aug 2 10:09:22 2022] POST /identity/v3/users => generated 288 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:09:22.868323 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045336fb-a26e-4c00-9c92-e1fdc9ac593f admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:09:22.879078 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045336fb-a26e-4c00-9c92-e1fdc9ac593f admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:09:22.883795 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b7445d-173a-4746-b30b-22c3c92c0bfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:09:22.884521 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b7445d-173a-4746-b30b-22c3c92c0bfd admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:22.884812 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b7445d-173a-4746-b30b-22c3c92c0bfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:22.885088 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b7445d-173a-4746-b30b-22c3c92c0bfd admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:22.885834 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b7445d-173a-4746-b30b-22c3c92c0bfd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:09:22.887456 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b7445d-173a-4746-b30b-22c3c92c0bfd admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:09:22.898905 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1218/2437] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:09:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:09:22.904925 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad7c2de-c293-4c4c-a2fc-a6db0c294424 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:09:22.921523 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad7c2de-c293-4c4c-a2fc-a6db0c294424 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:09:22.927172 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bf5278-b37e-4573-98e8-3c2e798e8be3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:09:22.927956 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bf5278-b37e-4573-98e8-3c2e798e8be3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:22.928176 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bf5278-b37e-4573-98e8-3c2e798e8be3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:22.928766 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bf5278-b37e-4573-98e8-3c2e798e8be3 admin admin] PATH_INFO: `/v3/projects/719b5b32ae654bbea7c88f258d9156b3/users/5a06762ab8da4d5b949cbb5dd30e413e/roles/1000cf318fa14915b8091599eaead5de` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:22.944792 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bf5278-b37e-4573-98e8-3c2e798e8be3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=719b5b32ae654bbea7c88f258d9156b3, user_id=5a06762ab8da4d5b949cbb5dd30e413e, role_id=1000cf318fa14915b8091599eaead5de)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:09:22.947493 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bf5278-b37e-4573-98e8-3c2e798e8be3 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:09:22.964364 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1220/2438] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 10:09:22 2022] PUT /identity/v3/projects/719b5b32ae654bbea7c88f258d9156b3/users/5a06762ab8da4d5b949cbb5dd30e413e/roles/1000cf318fa14915b8091599eaead5de => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:09:22.968456 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b7445d-173a-4746-b30b-22c3c92c0bfd admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:09:22.979680 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b7445d-173a-4746-b30b-22c3c92c0bfd admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:09:23.038555 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a158286f-10a5-4f24-88b6-fed397d89c71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:09:23.039182 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a158286f-10a5-4f24-88b6-fed397d89c71 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:23.039467 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a158286f-10a5-4f24-88b6-fed397d89c71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:23.039744 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a158286f-10a5-4f24-88b6-fed397d89c71 admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:23.040482 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a158286f-10a5-4f24-88b6-fed397d89c71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:09:23.042706 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a158286f-10a5-4f24-88b6-fed397d89c71 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:09:23.053218 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1219/2439] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:09:22 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:09:23.057458 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bf5278-b37e-4573-98e8-3c2e798e8be3 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:09:23.068108 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bf5278-b37e-4573-98e8-3c2e798e8be3 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:09:23.072840 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadb4021-d460-4b86-8a87-45e39a026c7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:09:23.073501 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadb4021-d460-4b86-8a87-45e39a026c7b admin admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:23.073787 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadb4021-d460-4b86-8a87-45e39a026c7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:23.074065 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadb4021-d460-4b86-8a87-45e39a026c7b admin admin] PATH_INFO: `/v3/projects/719b5b32ae654bbea7c88f258d9156b3/users/5a06762ab8da4d5b949cbb5dd30e413e/roles/1000cf318fa14915b8091599eaead5de` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:23.076057 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadb4021-d460-4b86-8a87-45e39a026c7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=719b5b32ae654bbea7c88f258d9156b3, user_id=5a06762ab8da4d5b949cbb5dd30e413e, role_id=1000cf318fa14915b8091599eaead5de)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:09:23.078746 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadb4021-d460-4b86-8a87-45e39a026c7b admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:09:23.089857 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1221/2440] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 10:09:23 2022] PUT /identity/v3/projects/719b5b32ae654bbea7c88f258d9156b3/users/5a06762ab8da4d5b949cbb5dd30e413e/roles/1000cf318fa14915b8091599eaead5de => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:09:23.099991 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5fe8db-976d-474b-a0e1-f178bef3697f None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:23.100301 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5fe8db-976d-474b-a0e1-f178bef3697f None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:23.100703 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5fe8db-976d-474b-a0e1-f178bef3697f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:23.129287 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-ac5fe8db-976d-474b-a0e1-f178bef3697f None None] MFA Rules not processed for user `5a06762ab8da4d5b949cbb5dd30e413e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:09:23.130521 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-ac5fe8db-976d-474b-a0e1-f178bef3697f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:09:23.201479 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1220/2441] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:09:23 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:09:23.212832 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadb4021-d460-4b86-8a87-45e39a026c7b admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:09:23.215318 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-aadb4021-d460-4b86-8a87-45e39a026c7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:09:23.279311 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadb4021-d460-4b86-8a87-45e39a026c7b admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:09:23.283111 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10da6b05-e217-4bb0-9db4-9dc80ab7f96c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d7de62b9ee4039bdc79627ac7fb499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:09:23.283694 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10da6b05-e217-4bb0-9db4-9dc80ab7f96c service cinder] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:23.283974 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10da6b05-e217-4bb0-9db4-9dc80ab7f96c service cinder] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:23.284269 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10da6b05-e217-4bb0-9db4-9dc80ab7f96c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:23.294000 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10da6b05-e217-4bb0-9db4-9dc80ab7f96c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:09:23.295444 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10da6b05-e217-4bb0-9db4-9dc80ab7f96c service cinder] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:09:23.305652 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1222/2442] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:09:23 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:09:23.384764 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5fe8db-976d-474b-a0e1-f178bef3697f None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:09:23.395002 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5fe8db-976d-474b-a0e1-f178bef3697f None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:09:23.398580 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03df38b0-bd7b-4acd-954d-ddfd03beab62 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d7de62b9ee4039bdc79627ac7fb499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:09:23.399188 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03df38b0-bd7b-4acd-954d-ddfd03beab62 service cinder] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:23.399473 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03df38b0-bd7b-4acd-954d-ddfd03beab62 service cinder] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:23.399745 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03df38b0-bd7b-4acd-954d-ddfd03beab62 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:23.434604 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:09:23.458816 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03df38b0-bd7b-4acd-954d-ddfd03beab62 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:09:23.460174 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03df38b0-bd7b-4acd-954d-ddfd03beab62 service cinder] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:09:23.470154 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1221/2443] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:09:23 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:09:32.551149 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10da6b05-e217-4bb0-9db4-9dc80ab7f96c service cinder] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:09:32.552564 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-10da6b05-e217-4bb0-9db4-9dc80ab7f96c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:09:32.597321 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:09:32.615194 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10da6b05-e217-4bb0-9db4-9dc80ab7f96c service cinder] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:09:32.618908 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6044b47e-a1a6-4d1e-ad4a-ca0528abe457 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:09:32.619539 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6044b47e-a1a6-4d1e-ad4a-ca0528abe457 service neutron] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:32.619823 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6044b47e-a1a6-4d1e-ad4a-ca0528abe457 service neutron] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:32.620099 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6044b47e-a1a6-4d1e-ad4a-ca0528abe457 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:32.621896 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-6044b47e-a1a6-4d1e-ad4a-ca0528abe457 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:09:32.723195 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6044b47e-a1a6-4d1e-ad4a-ca0528abe457 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:09:32.724759 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6044b47e-a1a6-4d1e-ad4a-ca0528abe457 service neutron] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:09:32.739301 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1223/2444] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:09:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:09:33.465877 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=11 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:09:33.975212 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03df38b0-bd7b-4acd-954d-ddfd03beab62 service cinder] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:09:33.985223 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03df38b0-bd7b-4acd-954d-ddfd03beab62 service cinder] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:09:33.989570 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d785a06f-3343-40b5-b3b4-79d46003a03f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:09:33.990184 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d785a06f-3343-40b5-b3b4-79d46003a03f service neutron] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:33.990466 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d785a06f-3343-40b5-b3b4-79d46003a03f service neutron] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:33.990739 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d785a06f-3343-40b5-b3b4-79d46003a03f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:33.992536 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-d785a06f-3343-40b5-b3b4-79d46003a03f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:09:34.010074 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d785a06f-3343-40b5-b3b4-79d46003a03f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:09:34.012558 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d785a06f-3343-40b5-b3b4-79d46003a03f service neutron] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:09:34.028353 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1222/2445] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:09:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:09:35.028155 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6044b47e-a1a6-4d1e-ad4a-ca0528abe457 service neutron] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:09:35.135573 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6044b47e-a1a6-4d1e-ad4a-ca0528abe457 service neutron] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:09:35.144156 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2096d1-16fa-4cbe-b376-2c19ceb98d64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:09:35.145283 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2096d1-16fa-4cbe-b376-2c19ceb98d64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:35.145779 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2096d1-16fa-4cbe-b376-2c19ceb98d64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:35.146316 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2096d1-16fa-4cbe-b376-2c19ceb98d64 admin admin] PATH_INFO: `/v3/users/ef4d2afb79b94ca58010725e332e44d7` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:35.148383 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2096d1-16fa-4cbe-b376-2c19ceb98d64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef4d2afb79b94ca58010725e332e44d7)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:09:35.152459 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2096d1-16fa-4cbe-b376-2c19ceb98d64 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:09:35.238781 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-7f2096d1-16fa-4cbe-b376-2c19ceb98d64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef4d2afb79b94ca58010725e332e44d7'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:09:35.249293 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-7f2096d1-16fa-4cbe-b376-2c19ceb98d64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef4d2afb79b94ca58010725e332e44d7'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:09:35.261679 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-7f2096d1-16fa-4cbe-b376-2c19ceb98d64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef4d2afb79b94ca58010725e332e44d7'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:09:35.262813 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-7f2096d1-16fa-4cbe-b376-2c19ceb98d64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef4d2afb79b94ca58010725e332e44d7'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:09:35.263774 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-7f2096d1-16fa-4cbe-b376-2c19ceb98d64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef4d2afb79b94ca58010725e332e44d7'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:09:35.271113 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:09:35.292033 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1224/2446] 10.222.0.64 () {58 vars in 1224 bytes} [Tue Aug 2 10:09:35 2022] DELETE /identity/v3/users/ef4d2afb79b94ca58010725e332e44d7 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:09:35.297303 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7657b5-5078-4612-a446-67eb95fceaa0 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:35.298194 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7657b5-5078-4612-a446-67eb95fceaa0 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:35.298629 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7657b5-5078-4612-a446-67eb95fceaa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:35.324711 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-6d7657b5-5078-4612-a446-67eb95fceaa0 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:09:35.326038 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-6d7657b5-5078-4612-a446-67eb95fceaa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:09:35.404145 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1223/2447] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:09:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:09:35.413229 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2096d1-16fa-4cbe-b376-2c19ceb98d64 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:09:35.414958 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-7f2096d1-16fa-4cbe-b376-2c19ceb98d64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:09:35.516822 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2096d1-16fa-4cbe-b376-2c19ceb98d64 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:09:35.522190 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d4d3a2-ce13-4838-a9c0-5ed8013f3a30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:09:35.522978 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d4d3a2-ce13-4838-a9c0-5ed8013f3a30 service neutron] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:35.523375 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d4d3a2-ce13-4838-a9c0-5ed8013f3a30 service neutron] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:35.523719 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d4d3a2-ce13-4838-a9c0-5ed8013f3a30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:35.536463 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d4d3a2-ce13-4838-a9c0-5ed8013f3a30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:09:35.538267 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d4d3a2-ce13-4838-a9c0-5ed8013f3a30 service neutron] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:09:35.551061 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1225/2448] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:09:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:09:36.013432 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7657b5-5078-4612-a446-67eb95fceaa0 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:09:36.015240 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-6d7657b5-5078-4612-a446-67eb95fceaa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:09:36.035882 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7657b5-5078-4612-a446-67eb95fceaa0 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:09:36.041672 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f6b7ad-00a5-4e97-9c39-25321eaebeed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:09:36.042710 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f6b7ad-00a5-4e97-9c39-25321eaebeed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:36.043433 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f6b7ad-00a5-4e97-9c39-25321eaebeed admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:36.044053 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f6b7ad-00a5-4e97-9c39-25321eaebeed admin admin] PATH_INFO: `/v3/projects/7ac1d1647c844d6e8d8037ca1cd4431f` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:36.046148 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f6b7ad-00a5-4e97-9c39-25321eaebeed admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ac1d1647c844d6e8d8037ca1cd4431f)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:09:36.048599 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f6b7ad-00a5-4e97-9c39-25321eaebeed admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:09:36.131328 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-e7f6b7ad-00a5-4e97-9c39-25321eaebeed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ac1d1647c844d6e8d8037ca1cd4431f', 'request_id': 'req-e7f6b7ad-00a5-4e97-9c39-25321eaebeed'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:09:36.138773 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-e7f6b7ad-00a5-4e97-9c39-25321eaebeed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ac1d1647c844d6e8d8037ca1cd4431f', 'request_id': 'req-e7f6b7ad-00a5-4e97-9c39-25321eaebeed'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:09:36.154434 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-e7f6b7ad-00a5-4e97-9c39-25321eaebeed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ac1d1647c844d6e8d8037ca1cd4431f', 'request_id': 'req-e7f6b7ad-00a5-4e97-9c39-25321eaebeed'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:09:36.189548 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-e7f6b7ad-00a5-4e97-9c39-25321eaebeed admin admin] The token cache is being invalidate because project 7ac1d1647c844d6e8d8037ca1cd4431f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80154) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:09:36.189939 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-e7f6b7ad-00a5-4e97-9c39-25321eaebeed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:09:36.191111 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-e7f6b7ad-00a5-4e97-9c39-25321eaebeed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:09:36.193290 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1224/2449] 10.222.0.64 () {58 vars in 1233 bytes} [Tue Aug 2 10:09:36 2022] DELETE /identity/v3/projects/7ac1d1647c844d6e8d8037ca1cd4431f => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:09:36.197320 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d4d3a2-ce13-4838-a9c0-5ed8013f3a30 service neutron] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:09:36.208086 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d4d3a2-ce13-4838-a9c0-5ed8013f3a30 service neutron] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:09:36.284224 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a84f1f9-70bc-4e7e-ac8a-b6b80fb17853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:09:36.285237 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a84f1f9-70bc-4e7e-ac8a-b6b80fb17853 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:36.285237 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a84f1f9-70bc-4e7e-ac8a-b6b80fb17853 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:36.285456 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a84f1f9-70bc-4e7e-ac8a-b6b80fb17853 admin admin] PATH_INFO: `/v3/users/5a06762ab8da4d5b949cbb5dd30e413e` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:36.286623 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a84f1f9-70bc-4e7e-ac8a-b6b80fb17853 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a06762ab8da4d5b949cbb5dd30e413e)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:09:36.288655 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a84f1f9-70bc-4e7e-ac8a-b6b80fb17853 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:09:36.336355 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-3a84f1f9-70bc-4e7e-ac8a-b6b80fb17853 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a06762ab8da4d5b949cbb5dd30e413e'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:09:36.343465 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-3a84f1f9-70bc-4e7e-ac8a-b6b80fb17853 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a06762ab8da4d5b949cbb5dd30e413e'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:09:36.351990 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-3a84f1f9-70bc-4e7e-ac8a-b6b80fb17853 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a06762ab8da4d5b949cbb5dd30e413e'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:09:36.352697 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-3a84f1f9-70bc-4e7e-ac8a-b6b80fb17853 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a06762ab8da4d5b949cbb5dd30e413e'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:09:36.353249 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-3a84f1f9-70bc-4e7e-ac8a-b6b80fb17853 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a06762ab8da4d5b949cbb5dd30e413e'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:09:36.372274 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1226/2450] 10.222.0.64 () {58 vars in 1224 bytes} [Tue Aug 2 10:09:36 2022] DELETE /identity/v3/users/5a06762ab8da4d5b949cbb5dd30e413e => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:09:36.873556 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f6b7ad-00a5-4e97-9c39-25321eaebeed admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:09:36.877928 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-e7f6b7ad-00a5-4e97-9c39-25321eaebeed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:09:36.900109 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f6b7ad-00a5-4e97-9c39-25321eaebeed admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:09:36.975107 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8de6053-6198-4766-af3d-d323d3a9445e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:09:36.975926 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8de6053-6198-4766-af3d-d323d3a9445e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:36.976096 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8de6053-6198-4766-af3d-d323d3a9445e admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:36.976390 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8de6053-6198-4766-af3d-d323d3a9445e admin admin] PATH_INFO: `/v3/projects/719b5b32ae654bbea7c88f258d9156b3` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:36.977649 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8de6053-6198-4766-af3d-d323d3a9445e admin admin] RBAC: Authorizing `identity:delete_project(project_id=719b5b32ae654bbea7c88f258d9156b3)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:09:36.979648 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8de6053-6198-4766-af3d-d323d3a9445e admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:09:36.992986 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:09:37.039270 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-f8de6053-6198-4766-af3d-d323d3a9445e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '719b5b32ae654bbea7c88f258d9156b3', 'request_id': 'req-f8de6053-6198-4766-af3d-d323d3a9445e'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:09:37.045649 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-f8de6053-6198-4766-af3d-d323d3a9445e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '719b5b32ae654bbea7c88f258d9156b3', 'request_id': 'req-f8de6053-6198-4766-af3d-d323d3a9445e'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:09:37.052380 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-f8de6053-6198-4766-af3d-d323d3a9445e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '719b5b32ae654bbea7c88f258d9156b3', 'request_id': 'req-f8de6053-6198-4766-af3d-d323d3a9445e'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:09:37.067793 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-f8de6053-6198-4766-af3d-d323d3a9445e admin admin] The token cache is being invalidate because project 719b5b32ae654bbea7c88f258d9156b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80154) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:09:37.068214 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-f8de6053-6198-4766-af3d-d323d3a9445e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:09:37.069122 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-f8de6053-6198-4766-af3d-d323d3a9445e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:09:37.070832 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1225/2451] 10.222.0.64 () {58 vars in 1233 bytes} [Tue Aug 2 10:09:36 2022] DELETE /identity/v3/projects/719b5b32ae654bbea7c88f258d9156b3 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:09:37.075916 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c225cded-5d24-4b1d-8827-cde424ea2be2 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:37.076220 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c225cded-5d24-4b1d-8827-cde424ea2be2 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:37.076536 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c225cded-5d24-4b1d-8827-cde424ea2be2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:37.098419 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-c225cded-5d24-4b1d-8827-cde424ea2be2 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:09:37.099670 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-c225cded-5d24-4b1d-8827-cde424ea2be2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:09:37.104371 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:09:37.171924 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1227/2452] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:09:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:09:37.179874 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6d7837-5862-4d82-b175-d4a060746f22 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:37.180238 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6d7837-5862-4d82-b175-d4a060746f22 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:37.180688 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6d7837-5862-4d82-b175-d4a060746f22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:37.202618 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-3c6d7837-5862-4d82-b175-d4a060746f22 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:09:37.204133 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-3c6d7837-5862-4d82-b175-d4a060746f22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:09:37.219247 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1226/2453] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:09:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:09:37.223579 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c225cded-5d24-4b1d-8827-cde424ea2be2 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:09:37.234497 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c225cded-5d24-4b1d-8827-cde424ea2be2 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:09:37.239522 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491281f8-a4aa-46b1-9ed2-a2b391d4fa91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:09:37.240176 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491281f8-a4aa-46b1-9ed2-a2b391d4fa91 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:37.240512 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491281f8-a4aa-46b1-9ed2-a2b391d4fa91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:37.240794 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491281f8-a4aa-46b1-9ed2-a2b391d4fa91 admin admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:37.241504 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491281f8-a4aa-46b1-9ed2-a2b391d4fa91 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:09:37.241818 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491281f8-a4aa-46b1-9ed2-a2b391d4fa91 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:09:37.243442 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491281f8-a4aa-46b1-9ed2-a2b391d4fa91 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:09:37.255692 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1228/2454] 10.222.0.64 () {58 vars in 1153 bytes} [Tue Aug 2 10:09:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:09:37.260365 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6d7837-5862-4d82-b175-d4a060746f22 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:09:37.270638 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6d7837-5862-4d82-b175-d4a060746f22 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:09:37.275680 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b86fc4-5ae5-4eaf-89e4-5c27adbbb0a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:09:37.276358 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b86fc4-5ae5-4eaf-89e4-5c27adbbb0a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:37.276651 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b86fc4-5ae5-4eaf-89e4-5c27adbbb0a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:37.277464 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b86fc4-5ae5-4eaf-89e4-5c27adbbb0a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:37.278172 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b86fc4-5ae5-4eaf-89e4-5c27adbbb0a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:09:37.280146 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b86fc4-5ae5-4eaf-89e4-5c27adbbb0a7 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:09:37.323084 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1227/2455] 10.222.0.64 () {60 vars in 1153 bytes} [Tue Aug 2 10:09:37 2022] POST /identity/v3/projects => generated 377 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:09:37.328109 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491281f8-a4aa-46b1-9ed2-a2b391d4fa91 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:09:37.340950 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491281f8-a4aa-46b1-9ed2-a2b391d4fa91 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:09:37.409085 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad71374f-8e17-4fb3-9389-a11a21afd5af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:09:37.409868 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad71374f-8e17-4fb3-9389-a11a21afd5af admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:37.410219 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad71374f-8e17-4fb3-9389-a11a21afd5af admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:37.410524 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad71374f-8e17-4fb3-9389-a11a21afd5af admin admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:37.411195 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad71374f-8e17-4fb3-9389-a11a21afd5af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:09:37.413195 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad71374f-8e17-4fb3-9389-a11a21afd5af admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:09:37.435192 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1229/2456] 10.222.0.64 () {60 vars in 1144 bytes} [Tue Aug 2 10:09:37 2022] POST /identity/v3/users => generated 289 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:09:37.439694 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b86fc4-5ae5-4eaf-89e4-5c27adbbb0a7 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:09:37.451972 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b86fc4-5ae5-4eaf-89e4-5c27adbbb0a7 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:09:37.457211 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e339428b-5bd1-4338-8bc4-5a31f2eeaba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:09:37.457825 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e339428b-5bd1-4338-8bc4-5a31f2eeaba9 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:37.458108 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e339428b-5bd1-4338-8bc4-5a31f2eeaba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:37.458383 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e339428b-5bd1-4338-8bc4-5a31f2eeaba9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:37.459096 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e339428b-5bd1-4338-8bc4-5a31f2eeaba9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:09:37.461106 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e339428b-5bd1-4338-8bc4-5a31f2eeaba9 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:09:37.472964 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1228/2457] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:09:37 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:09:37.481648 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad71374f-8e17-4fb3-9389-a11a21afd5af admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:09:37.492523 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad71374f-8e17-4fb3-9389-a11a21afd5af admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:09:37.499085 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860f17a5-304e-47de-ba40-49f1432cfce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:09:37.500059 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860f17a5-304e-47de-ba40-49f1432cfce1 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:37.500492 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860f17a5-304e-47de-ba40-49f1432cfce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:37.500963 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860f17a5-304e-47de-ba40-49f1432cfce1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:37.502054 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860f17a5-304e-47de-ba40-49f1432cfce1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:09:37.504359 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860f17a5-304e-47de-ba40-49f1432cfce1 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:09:37.520434 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1230/2458] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:09:37 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:09:37.521672 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e339428b-5bd1-4338-8bc4-5a31f2eeaba9 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:09:37.534248 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e339428b-5bd1-4338-8bc4-5a31f2eeaba9 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:09:37.543857 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f8c2a9-5b15-4f4d-a05b-49376112c292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:09:37.544586 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f8c2a9-5b15-4f4d-a05b-49376112c292 admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:37.544895 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f8c2a9-5b15-4f4d-a05b-49376112c292 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:37.545175 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f8c2a9-5b15-4f4d-a05b-49376112c292 admin admin] PATH_INFO: `/v3/projects/8b4310b16f654035af4bb5b131e25aaa/users/95378fc2851e4134ab8f1af34d497092/roles/66c1074780954f8e87ed11a5c4e8c36a` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:37.566269 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f8c2a9-5b15-4f4d-a05b-49376112c292 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b4310b16f654035af4bb5b131e25aaa, user_id=95378fc2851e4134ab8f1af34d497092, role_id=66c1074780954f8e87ed11a5c4e8c36a)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:09:37.566980 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f8c2a9-5b15-4f4d-a05b-49376112c292 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:09:37.581723 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1229/2459] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 10:09:37 2022] PUT /identity/v3/projects/8b4310b16f654035af4bb5b131e25aaa/users/95378fc2851e4134ab8f1af34d497092/roles/66c1074780954f8e87ed11a5c4e8c36a => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:09:37.594198 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5b8c93-6dc7-471e-bd5f-75e7c0a5dc55 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:37.594493 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5b8c93-6dc7-471e-bd5f-75e7c0a5dc55 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:37.594775 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5b8c93-6dc7-471e-bd5f-75e7c0a5dc55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:37.626845 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-ba5b8c93-6dc7-471e-bd5f-75e7c0a5dc55 None None] MFA Rules not processed for user `95378fc2851e4134ab8f1af34d497092`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:09:37.627475 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-ba5b8c93-6dc7-471e-bd5f-75e7c0a5dc55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:09:37.696789 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1231/2460] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:09:37 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:09:37.709332 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f8c2a9-5b15-4f4d-a05b-49376112c292 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:09:37.711256 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-97f8c2a9-5b15-4f4d-a05b-49376112c292 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:09:37.778215 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f8c2a9-5b15-4f4d-a05b-49376112c292 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:09:37.782033 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c7d1ef-720b-402e-8764-8600c49efd1e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d7de62b9ee4039bdc79627ac7fb499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:09:37.782653 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c7d1ef-720b-402e-8764-8600c49efd1e service cinder] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:37.782936 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c7d1ef-720b-402e-8764-8600c49efd1e service cinder] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:37.783211 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c7d1ef-720b-402e-8764-8600c49efd1e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:37.792219 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c7d1ef-720b-402e-8764-8600c49efd1e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:09:37.793723 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c7d1ef-720b-402e-8764-8600c49efd1e service cinder] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:09:37.802913 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1230/2461] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:09:37 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:09:40.133547 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d315b035-83cd-4bde-a281-e61f94a42cd2 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:40.134142 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d315b035-83cd-4bde-a281-e61f94a42cd2 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:40.134142 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d315b035-83cd-4bde-a281-e61f94a42cd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:40.140404 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:09:40.182915 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-d315b035-83cd-4bde-a281-e61f94a42cd2 None None] MFA Rules not processed for user `dfe32e33711e42118f3ff5021762604f`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:09:40.184148 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-d315b035-83cd-4bde-a281-e61f94a42cd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:09:40.250261 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1232/2462] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:09:40 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:09:42.759522 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c7d1ef-720b-402e-8764-8600c49efd1e service cinder] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:09:42.761105 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-02c7d1ef-720b-402e-8764-8600c49efd1e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:09:42.797100 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:09:42.841414 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c7d1ef-720b-402e-8764-8600c49efd1e service cinder] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:09:42.847034 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d892bdf-b33d-4ded-b2a7-94eabf5659a5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99655e1f9ca24472a86cd26b0dbd1fc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:09:42.847671 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d892bdf-b33d-4ded-b2a7-94eabf5659a5 service nova] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:09:42.847960 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d892bdf-b33d-4ded-b2a7-94eabf5659a5 service nova] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:09:42.848270 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d892bdf-b33d-4ded-b2a7-94eabf5659a5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:09:42.850081 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-7d892bdf-b33d-4ded-b2a7-94eabf5659a5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:09:42.941233 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d892bdf-b33d-4ded-b2a7-94eabf5659a5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:09:42.942829 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d892bdf-b33d-4ded-b2a7-94eabf5659a5 service nova] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:09:42.961712 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1231/2463] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:09:42 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:09:50.242855 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=32 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:09:52.958983 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=55 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:10:01.524440 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d315b035-83cd-4bde-a281-e61f94a42cd2 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:10:01.525890 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-d315b035-83cd-4bde-a281-e61f94a42cd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:10:01.547231 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f1dc18-a05e-423b-8d21-a65a42263c79 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:10:01.547783 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f1dc18-a05e-423b-8d21-a65a42263c79 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:10:01.547783 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f1dc18-a05e-423b-8d21-a65a42263c79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:10:01.567281 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-04f1dc18-a05e-423b-8d21-a65a42263c79 None None] MFA Rules not processed for user `7739a00c6524467d8fd139183c6667cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:10:01.568540 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-04f1dc18-a05e-423b-8d21-a65a42263c79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:10:01.586175 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d315b035-83cd-4bde-a281-e61f94a42cd2 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:10:01.589807 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fa82ba-1e16-4ee5-b43d-9e6092108c75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:10:01.590404 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fa82ba-1e16-4ee5-b43d-9e6092108c75 service neutron] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:10:01.590694 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fa82ba-1e16-4ee5-b43d-9e6092108c75 service neutron] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:10:01.590970 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fa82ba-1e16-4ee5-b43d-9e6092108c75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:10:01.592764 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-32fa82ba-1e16-4ee5-b43d-9e6092108c75 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:10:01.603067 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fa82ba-1e16-4ee5-b43d-9e6092108c75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:10:01.604565 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fa82ba-1e16-4ee5-b43d-9e6092108c75 service neutron] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:10:01.616477 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1233/2464] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:10:01 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:10:01.629264 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1232/2465] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:10:01 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:10:02.503938 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fa82ba-1e16-4ee5-b43d-9e6092108c75 service neutron] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:10:02.507791 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-32fa82ba-1e16-4ee5-b43d-9e6092108c75 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:10:02.581772 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fa82ba-1e16-4ee5-b43d-9e6092108c75 service neutron] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:10:02.586935 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada5a87e-3d31-4c05-bf7d-7dda9492c382 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b770f073c86b4d88837a8b1f796a8b1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:10:02.587560 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada5a87e-3d31-4c05-bf7d-7dda9492c382 service placement] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:10:02.587849 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada5a87e-3d31-4c05-bf7d-7dda9492c382 service placement] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:10:02.588129 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada5a87e-3d31-4c05-bf7d-7dda9492c382 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:10:02.590500 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-ada5a87e-3d31-4c05-bf7d-7dda9492c382 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:10:02.601921 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada5a87e-3d31-4c05-bf7d-7dda9492c382 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:10:02.603530 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada5a87e-3d31-4c05-bf7d-7dda9492c382 service placement] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:10:02.614474 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1234/2466] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:10:02 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:10:11.622322 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=25 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:10:12.610145 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=58 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:10:14.199232 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9508d6ea-751f-4115-8275-61a48c9fbb97 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:10:14.199232 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9508d6ea-751f-4115-8275-61a48c9fbb97 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:10:14.199232 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9508d6ea-751f-4115-8275-61a48c9fbb97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:10:14.258730 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-9508d6ea-751f-4115-8275-61a48c9fbb97 None None] MFA Rules not processed for user `5338bc8c594e4bc9bc41019226bf8d87`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:10:14.259726 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-9508d6ea-751f-4115-8275-61a48c9fbb97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:10:14.338509 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1233/2467] 10.222.0.64 () {58 vars in 1009 bytes} [Tue Aug 2 10:10:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:10:14.390140 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada5a87e-3d31-4c05-bf7d-7dda9492c382 service placement] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:10:14.391707 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-ada5a87e-3d31-4c05-bf7d-7dda9492c382 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:10:14.522808 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada5a87e-3d31-4c05-bf7d-7dda9492c382 service placement] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:10:14.531927 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e91a52-5854-411d-8bae-9de14c4be6c3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a371e3df3764a4cbb1b8368aa74bd65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:10:14.533182 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e91a52-5854-411d-8bae-9de14c4be6c3 service swift] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:10:14.533802 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e91a52-5854-411d-8bae-9de14c4be6c3 service swift] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:10:14.534274 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e91a52-5854-411d-8bae-9de14c4be6c3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:10:14.553187 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e91a52-5854-411d-8bae-9de14c4be6c3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:10:14.554869 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e91a52-5854-411d-8bae-9de14c4be6c3 service swift] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:10:14.565177 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1235/2468] 10.222.0.64 () {60 vars in 1384 bytes} [Tue Aug 2 10:10:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 02 10:10:15.635193 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9508d6ea-751f-4115-8275-61a48c9fbb97 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:10:15.731292 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9508d6ea-751f-4115-8275-61a48c9fbb97 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:10:15.736756 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8133d4-cb6a-4cb4-8614-ac493d3003cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:10:15.737578 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8133d4-cb6a-4cb4-8614-ac493d3003cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:10:15.737866 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8133d4-cb6a-4cb4-8614-ac493d3003cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:10:15.738149 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8133d4-cb6a-4cb4-8614-ac493d3003cd admin admin] PATH_INFO: `/v3/users/95378fc2851e4134ab8f1af34d497092` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:10:15.739366 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8133d4-cb6a-4cb4-8614-ac493d3003cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=95378fc2851e4134ab8f1af34d497092)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:10:15.741421 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8133d4-cb6a-4cb4-8614-ac493d3003cd admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:10:15.818357 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-0f8133d4-cb6a-4cb4-8614-ac493d3003cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95378fc2851e4134ab8f1af34d497092'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:10:15.837657 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-0f8133d4-cb6a-4cb4-8614-ac493d3003cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95378fc2851e4134ab8f1af34d497092'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:10:15.838481 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-0f8133d4-cb6a-4cb4-8614-ac493d3003cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95378fc2851e4134ab8f1af34d497092'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:10:15.853780 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-0f8133d4-cb6a-4cb4-8614-ac493d3003cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95378fc2851e4134ab8f1af34d497092'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:10:15.868480 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:10:15.876621 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-0f8133d4-cb6a-4cb4-8614-ac493d3003cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95378fc2851e4134ab8f1af34d497092'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:10:15.891341 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1234/2469] 10.222.0.64 () {58 vars in 1224 bytes} [Tue Aug 2 10:10:15 2022] DELETE /identity/v3/users/95378fc2851e4134ab8f1af34d497092 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:10:15.892524 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1950553-3141-492c-8fa3-3bd3eb67f640 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:10:15.893459 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1950553-3141-492c-8fa3-3bd3eb67f640 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:10:15.893748 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1950553-3141-492c-8fa3-3bd3eb67f640 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:10:15.915014 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-a1950553-3141-492c-8fa3-3bd3eb67f640 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:10:15.916275 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-a1950553-3141-492c-8fa3-3bd3eb67f640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:10:16.012521 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1236/2470] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:10:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:10:16.021610 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8133d4-cb6a-4cb4-8614-ac493d3003cd admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:10:16.023182 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-0f8133d4-cb6a-4cb4-8614-ac493d3003cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:10:16.104398 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8133d4-cb6a-4cb4-8614-ac493d3003cd admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:10:16.108781 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0f8cc6-a394-4360-ad9c-06cc2f86f2c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:10:16.109662 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0f8cc6-a394-4360-ad9c-06cc2f86f2c1 service neutron] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:10:16.110137 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0f8cc6-a394-4360-ad9c-06cc2f86f2c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:10:16.110628 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0f8cc6-a394-4360-ad9c-06cc2f86f2c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:10:16.122789 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0f8cc6-a394-4360-ad9c-06cc2f86f2c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:10:16.124836 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0f8cc6-a394-4360-ad9c-06cc2f86f2c1 service neutron] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:10:16.137108 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1235/2471] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:10:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:10:16.469700 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1950553-3141-492c-8fa3-3bd3eb67f640 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:10:16.471128 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-a1950553-3141-492c-8fa3-3bd3eb67f640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:10:16.488136 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1950553-3141-492c-8fa3-3bd3eb67f640 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:10:16.494090 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d7047a-11c4-4085-a2e0-c0644dafe045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:10:16.495107 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d7047a-11c4-4085-a2e0-c0644dafe045 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:10:16.495619 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d7047a-11c4-4085-a2e0-c0644dafe045 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:10:16.496002 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d7047a-11c4-4085-a2e0-c0644dafe045 admin admin] PATH_INFO: `/v3/projects/8b4310b16f654035af4bb5b131e25aaa` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:10:16.497345 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d7047a-11c4-4085-a2e0-c0644dafe045 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b4310b16f654035af4bb5b131e25aaa)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:10:16.500493 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d7047a-11c4-4085-a2e0-c0644dafe045 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:10:16.570218 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-49d7047a-11c4-4085-a2e0-c0644dafe045 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b4310b16f654035af4bb5b131e25aaa', 'request_id': 'req-49d7047a-11c4-4085-a2e0-c0644dafe045'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:10:16.577265 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-49d7047a-11c4-4085-a2e0-c0644dafe045 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b4310b16f654035af4bb5b131e25aaa', 'request_id': 'req-49d7047a-11c4-4085-a2e0-c0644dafe045'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:10:16.585056 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-49d7047a-11c4-4085-a2e0-c0644dafe045 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b4310b16f654035af4bb5b131e25aaa', 'request_id': 'req-49d7047a-11c4-4085-a2e0-c0644dafe045'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:10:16.593001 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:10:16.618464 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-49d7047a-11c4-4085-a2e0-c0644dafe045 admin admin] The token cache is being invalidate because project 8b4310b16f654035af4bb5b131e25aaa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80153) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:10:16.619460 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-49d7047a-11c4-4085-a2e0-c0644dafe045 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:10:16.620380 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-49d7047a-11c4-4085-a2e0-c0644dafe045 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:10:16.622337 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1237/2472] 10.222.0.64 () {58 vars in 1233 bytes} [Tue Aug 2 10:10:16 2022] DELETE /identity/v3/projects/8b4310b16f654035af4bb5b131e25aaa => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:10:16.629206 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81edd9a-ac17-47f7-90ab-cb59c37e34de None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:10:16.629596 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81edd9a-ac17-47f7-90ab-cb59c37e34de None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:10:16.629980 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81edd9a-ac17-47f7-90ab-cb59c37e34de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:10:16.652561 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-e81edd9a-ac17-47f7-90ab-cb59c37e34de None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:10:16.653857 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-e81edd9a-ac17-47f7-90ab-cb59c37e34de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:10:16.723089 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1236/2473] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:10:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:10:16.732268 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937bb82a-7538-4901-8957-7526fdd3b244 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:10:16.732729 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937bb82a-7538-4901-8957-7526fdd3b244 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:10:16.733304 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937bb82a-7538-4901-8957-7526fdd3b244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:10:16.759363 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-937bb82a-7538-4901-8957-7526fdd3b244 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:10:16.762677 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-937bb82a-7538-4901-8957-7526fdd3b244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:10:16.771443 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1238/2474] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:10:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:10:16.775980 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81edd9a-ac17-47f7-90ab-cb59c37e34de None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:10:16.789672 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81edd9a-ac17-47f7-90ab-cb59c37e34de None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:10:16.794560 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a62ebb-4106-421f-b2fa-21da14be7a28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:10:16.795818 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a62ebb-4106-421f-b2fa-21da14be7a28 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:10:16.796065 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a62ebb-4106-421f-b2fa-21da14be7a28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:10:16.796380 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a62ebb-4106-421f-b2fa-21da14be7a28 admin admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:10:16.797098 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a62ebb-4106-421f-b2fa-21da14be7a28 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:10:16.797409 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a62ebb-4106-421f-b2fa-21da14be7a28 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:10:16.799012 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a62ebb-4106-421f-b2fa-21da14be7a28 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:10:16.812303 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1237/2475] 10.222.0.64 () {58 vars in 1153 bytes} [Tue Aug 2 10:10:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:10:16.816648 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937bb82a-7538-4901-8957-7526fdd3b244 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:10:16.830626 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937bb82a-7538-4901-8957-7526fdd3b244 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:10:16.841275 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17aa98c-baf1-4c5c-95b8-3deb4df43add admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:10:16.842788 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17aa98c-baf1-4c5c-95b8-3deb4df43add admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:10:16.843280 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17aa98c-baf1-4c5c-95b8-3deb4df43add admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:10:16.843776 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17aa98c-baf1-4c5c-95b8-3deb4df43add admin admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:10:16.844704 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17aa98c-baf1-4c5c-95b8-3deb4df43add admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:10:16.852213 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17aa98c-baf1-4c5c-95b8-3deb4df43add admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:10:16.894874 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1239/2476] 10.222.0.64 () {60 vars in 1153 bytes} [Tue Aug 2 10:10:16 2022] POST /identity/v3/projects => generated 383 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:10:16.896701 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a62ebb-4106-421f-b2fa-21da14be7a28 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:10:16.909186 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a62ebb-4106-421f-b2fa-21da14be7a28 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:10:16.999061 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9035bbb9-5d17-4a7d-9149-da38da3b2a65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:10:16.999793 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9035bbb9-5d17-4a7d-9149-da38da3b2a65 admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:10:17.000143 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9035bbb9-5d17-4a7d-9149-da38da3b2a65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:10:17.000518 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9035bbb9-5d17-4a7d-9149-da38da3b2a65 admin admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:10:17.001742 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9035bbb9-5d17-4a7d-9149-da38da3b2a65 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:10:17.003345 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9035bbb9-5d17-4a7d-9149-da38da3b2a65 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:10:17.032364 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1238/2477] 10.222.0.64 () {60 vars in 1144 bytes} [Tue Aug 2 10:10:16 2022] POST /identity/v3/users => generated 292 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:10:17.033284 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17aa98c-baf1-4c5c-95b8-3deb4df43add admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:10:17.049846 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17aa98c-baf1-4c5c-95b8-3deb4df43add admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:10:17.055654 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a838f7b-b9de-4861-b355-9978292c7421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:10:17.056304 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a838f7b-b9de-4861-b355-9978292c7421 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:10:17.056592 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a838f7b-b9de-4861-b355-9978292c7421 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:10:17.057480 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a838f7b-b9de-4861-b355-9978292c7421 admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:10:17.058222 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a838f7b-b9de-4861-b355-9978292c7421 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:10:17.059851 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a838f7b-b9de-4861-b355-9978292c7421 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:10:17.080894 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9035bbb9-5d17-4a7d-9149-da38da3b2a65 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:10:17.082242 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1240/2478] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:10:17 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:10:17.091737 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9035bbb9-5d17-4a7d-9149-da38da3b2a65 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:10:17.097119 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306ae4b7-40d3-4733-9ebb-9644a53f283a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:10:17.098255 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306ae4b7-40d3-4733-9ebb-9644a53f283a admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:10:17.099263 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306ae4b7-40d3-4733-9ebb-9644a53f283a admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:10:17.099551 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306ae4b7-40d3-4733-9ebb-9644a53f283a admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:10:17.100328 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306ae4b7-40d3-4733-9ebb-9644a53f283a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:10:17.101952 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306ae4b7-40d3-4733-9ebb-9644a53f283a admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:10:17.110496 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:10:17.121784 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1239/2479] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:10:17 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:10:17.122319 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a838f7b-b9de-4861-b355-9978292c7421 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:10:17.140532 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a838f7b-b9de-4861-b355-9978292c7421 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:10:17.148319 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f362b1cb-3a38-41e9-99fe-9d3158da4882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:10:17.149012 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f362b1cb-3a38-41e9-99fe-9d3158da4882 admin admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:10:17.149289 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f362b1cb-3a38-41e9-99fe-9d3158da4882 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:10:17.149634 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f362b1cb-3a38-41e9-99fe-9d3158da4882 admin admin] PATH_INFO: `/v3/projects/25fd81d1b89c4a5788588d147622baed/users/c5cb1a83603e437ea576e127c8891fb5/roles/66c1074780954f8e87ed11a5c4e8c36a` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:10:17.170861 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f362b1cb-3a38-41e9-99fe-9d3158da4882 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25fd81d1b89c4a5788588d147622baed, user_id=c5cb1a83603e437ea576e127c8891fb5, role_id=66c1074780954f8e87ed11a5c4e8c36a)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:10:17.173579 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f362b1cb-3a38-41e9-99fe-9d3158da4882 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:10:17.188804 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1241/2480] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 10:10:17 2022] PUT /identity/v3/projects/25fd81d1b89c4a5788588d147622baed/users/c5cb1a83603e437ea576e127c8891fb5/roles/66c1074780954f8e87ed11a5c4e8c36a => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:10:17.194045 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f5112c-9515-490c-ba26-a441f9b712c6 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:10:17.194456 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f5112c-9515-490c-ba26-a441f9b712c6 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:10:17.194928 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f5112c-9515-490c-ba26-a441f9b712c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:10:17.220305 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-b6f5112c-9515-490c-ba26-a441f9b712c6 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:10:17.221541 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-b6f5112c-9515-490c-ba26-a441f9b712c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:10:17.289822 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1240/2481] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:10:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:10:17.300272 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f362b1cb-3a38-41e9-99fe-9d3158da4882 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:10:17.302346 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-f362b1cb-3a38-41e9-99fe-9d3158da4882 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:10:17.365724 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f362b1cb-3a38-41e9-99fe-9d3158da4882 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:10:17.369839 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ecd8c6-8808-43f2-854d-ac3261d90263 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:10:17.370522 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ecd8c6-8808-43f2-854d-ac3261d90263 service neutron] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:10:17.370815 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ecd8c6-8808-43f2-854d-ac3261d90263 service neutron] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:10:17.371280 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ecd8c6-8808-43f2-854d-ac3261d90263 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:10:17.387633 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ecd8c6-8808-43f2-854d-ac3261d90263 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:10:17.389822 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ecd8c6-8808-43f2-854d-ac3261d90263 service neutron] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:10:17.400428 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1242/2482] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:10:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:10:18.216943 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664ca600-f9fb-47f0-a1a9-764b25d29f91 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:10:18.218134 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664ca600-f9fb-47f0-a1a9-764b25d29f91 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:10:18.218134 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664ca600-f9fb-47f0-a1a9-764b25d29f91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:10:18.251327 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-664ca600-f9fb-47f0-a1a9-764b25d29f91 None None] MFA Rules not processed for user `c5cb1a83603e437ea576e127c8891fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:10:18.252582 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-664ca600-f9fb-47f0-a1a9-764b25d29f91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:10:18.361632 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1241/2483] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:10:18 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:10:18.373950 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ecd8c6-8808-43f2-854d-ac3261d90263 service neutron] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:10:18.375914 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-37ecd8c6-8808-43f2-854d-ac3261d90263 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:10:18.473390 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ecd8c6-8808-43f2-854d-ac3261d90263 service neutron] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:10:18.477311 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071b5f14-22bc-4e61-a489-75839b40d5c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d7de62b9ee4039bdc79627ac7fb499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:10:18.478041 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071b5f14-22bc-4e61-a489-75839b40d5c2 service cinder] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:10:18.478429 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071b5f14-22bc-4e61-a489-75839b40d5c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:10:18.478826 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071b5f14-22bc-4e61-a489-75839b40d5c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:10:18.490206 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071b5f14-22bc-4e61-a489-75839b40d5c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:10:18.491936 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071b5f14-22bc-4e61-a489-75839b40d5c2 service cinder] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:10:18.504554 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1243/2484] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:10:18 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:10:19.943894 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e183ad3f-19d2-4073-88d1-91f0e751d4e5 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:10:19.944473 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e183ad3f-19d2-4073-88d1-91f0e751d4e5 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:10:19.944473 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e183ad3f-19d2-4073-88d1-91f0e751d4e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:10:19.964571 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-e183ad3f-19d2-4073-88d1-91f0e751d4e5 None None] MFA Rules not processed for user `c5cb1a83603e437ea576e127c8891fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:10:19.966165 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-e183ad3f-19d2-4073-88d1-91f0e751d4e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:10:19.978973 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1242/2485] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:10:19 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:10:20.226013 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071b5f14-22bc-4e61-a489-75839b40d5c2 service cinder] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:10:20.238509 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071b5f14-22bc-4e61-a489-75839b40d5c2 service cinder] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:10:20.248550 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c374bee2-06d8-44f1-9c58-f3173422da78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:10:20.249178 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c374bee2-06d8-44f1-9c58-f3173422da78 service neutron] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:10:20.249466 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c374bee2-06d8-44f1-9c58-f3173422da78 service neutron] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:10:20.249746 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c374bee2-06d8-44f1-9c58-f3173422da78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:10:20.251525 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-c374bee2-06d8-44f1-9c58-f3173422da78 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:10:20.267469 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c374bee2-06d8-44f1-9c58-f3173422da78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:10:20.268995 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c374bee2-06d8-44f1-9c58-f3173422da78 service neutron] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:10:20.282755 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1244/2486] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:10:20 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:10:27.312223 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e183ad3f-19d2-4073-88d1-91f0e751d4e5 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:10:27.322171 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e183ad3f-19d2-4073-88d1-91f0e751d4e5 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:10:27.325734 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222a5051-88f6-40d0-b654-8a0283079439 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:10:27.326335 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222a5051-88f6-40d0-b654-8a0283079439 service neutron] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:10:27.326617 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222a5051-88f6-40d0-b654-8a0283079439 service neutron] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:10:27.326931 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222a5051-88f6-40d0-b654-8a0283079439 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:10:27.329353 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-222a5051-88f6-40d0-b654-8a0283079439 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:10:27.394420 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222a5051-88f6-40d0-b654-8a0283079439 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:10:27.395865 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222a5051-88f6-40d0-b654-8a0283079439 service neutron] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:10:27.405472 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1243/2487] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:10:27 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:10:30.273941 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:DELETE=1,keystone:SELECT=87,keystone:INSERT=2 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:10:33.538529 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c374bee2-06d8-44f1-9c58-f3173422da78 service neutron] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:10:33.540109 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-c374bee2-06d8-44f1-9c58-f3173422da78 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:10:33.636691 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c374bee2-06d8-44f1-9c58-f3173422da78 service neutron] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:10:33.642603 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f097ba-edee-474e-9e9a-b2dabb44ed78 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99655e1f9ca24472a86cd26b0dbd1fc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:10:33.643243 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f097ba-edee-474e-9e9a-b2dabb44ed78 service nova] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:10:33.643638 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f097ba-edee-474e-9e9a-b2dabb44ed78 service nova] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:10:33.643924 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f097ba-edee-474e-9e9a-b2dabb44ed78 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:10:33.646128 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-62f097ba-edee-474e-9e9a-b2dabb44ed78 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:10:33.659635 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f097ba-edee-474e-9e9a-b2dabb44ed78 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:10:33.661305 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f097ba-edee-474e-9e9a-b2dabb44ed78 service nova] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:10:33.673571 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1245/2488] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:10:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:10:35.604668 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222a5051-88f6-40d0-b654-8a0283079439 service neutron] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:10:35.606303 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-222a5051-88f6-40d0-b654-8a0283079439 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:10:35.635554 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:10:35.694357 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222a5051-88f6-40d0-b654-8a0283079439 service neutron] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:10:35.697548 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60a0d37-1557-4b4f-9a8e-597922718762 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04fb1dbe260a4721aaa0bd0493260d33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:10:35.698166 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60a0d37-1557-4b4f-9a8e-597922718762 service glance] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:10:35.698441 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60a0d37-1557-4b4f-9a8e-597922718762 service glance] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:10:35.698714 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60a0d37-1557-4b4f-9a8e-597922718762 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:10:35.700505 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-d60a0d37-1557-4b4f-9a8e-597922718762 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:10:35.714848 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60a0d37-1557-4b4f-9a8e-597922718762 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:10:35.716336 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60a0d37-1557-4b4f-9a8e-597922718762 service glance] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:10:35.728700 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1244/2489] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:10:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:10:35.811076 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f097ba-edee-474e-9e9a-b2dabb44ed78 service nova] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:10:35.820981 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f097ba-edee-474e-9e9a-b2dabb44ed78 service nova] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:10:35.824501 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48aff91b-1b8a-4ed3-90d6-5c05bc1d9c62 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:10:35.825104 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aff91b-1b8a-4ed3-90d6-5c05bc1d9c62 service neutron] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:10:35.825387 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aff91b-1b8a-4ed3-90d6-5c05bc1d9c62 service neutron] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:10:35.825660 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aff91b-1b8a-4ed3-90d6-5c05bc1d9c62 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:10:35.827387 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-48aff91b-1b8a-4ed3-90d6-5c05bc1d9c62 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:10:35.839424 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48aff91b-1b8a-4ed3-90d6-5c05bc1d9c62 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:10:35.840931 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48aff91b-1b8a-4ed3-90d6-5c05bc1d9c62 service neutron] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:10:35.849647 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1246/2490] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:10:35 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:10:35.922408 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60a0d37-1557-4b4f-9a8e-597922718762 service glance] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:10:35.931716 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60a0d37-1557-4b4f-9a8e-597922718762 service glance] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:10:35.935271 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415ffc25-4c47-43b5-90e2-29dc0ea27558 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:10:35.935870 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415ffc25-4c47-43b5-90e2-29dc0ea27558 service neutron] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:10:35.936173 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415ffc25-4c47-43b5-90e2-29dc0ea27558 service neutron] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:10:35.936495 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415ffc25-4c47-43b5-90e2-29dc0ea27558 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:10:35.938238 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-415ffc25-4c47-43b5-90e2-29dc0ea27558 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:10:35.951319 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415ffc25-4c47-43b5-90e2-29dc0ea27558 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:10:35.953276 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415ffc25-4c47-43b5-90e2-29dc0ea27558 service neutron] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:10:35.963930 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1245/2491] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:10:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:10:38.796415 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5523b150-8217-430e-b249-c7f39aed70bf None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:10:38.796999 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5523b150-8217-430e-b249-c7f39aed70bf None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:10:38.796999 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5523b150-8217-430e-b249-c7f39aed70bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:10:38.816617 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-5523b150-8217-430e-b249-c7f39aed70bf None None] MFA Rules not processed for user `5338bc8c594e4bc9bc41019226bf8d87`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:10:38.817835 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-5523b150-8217-430e-b249-c7f39aed70bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:10:38.882678 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1247/2492] 10.222.0.64 () {58 vars in 1009 bytes} [Tue Aug 2 10:10:38 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:10:38.895432 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415ffc25-4c47-43b5-90e2-29dc0ea27558 service neutron] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:10:38.896836 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-415ffc25-4c47-43b5-90e2-29dc0ea27558 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:10:38.966303 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415ffc25-4c47-43b5-90e2-29dc0ea27558 service neutron] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:10:38.972226 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da1a78e-e3f3-4095-ad16-11eaf81ebb28 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a371e3df3764a4cbb1b8368aa74bd65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:10:38.972876 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da1a78e-e3f3-4095-ad16-11eaf81ebb28 service swift] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:10:38.973162 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da1a78e-e3f3-4095-ad16-11eaf81ebb28 service swift] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:10:38.973437 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da1a78e-e3f3-4095-ad16-11eaf81ebb28 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:10:38.982515 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da1a78e-e3f3-4095-ad16-11eaf81ebb28 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:10:38.984385 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da1a78e-e3f3-4095-ad16-11eaf81ebb28 service swift] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:10:38.993721 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1246/2493] 10.222.0.64 () {60 vars in 1384 bytes} [Tue Aug 2 10:10:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 02 10:10:48.876666 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=65 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:10:48.990311 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=65 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:11:12.278083 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5523b150-8217-430e-b249-c7f39aed70bf None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:11:12.279497 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-5523b150-8217-430e-b249-c7f39aed70bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:11:12.291321 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5523b150-8217-430e-b249-c7f39aed70bf None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:11:12.295076 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1519946-71e8-4acc-b301-e7aabfc6490a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d7de62b9ee4039bdc79627ac7fb499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:11:12.295696 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1519946-71e8-4acc-b301-e7aabfc6490a service cinder] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:11:12.295989 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1519946-71e8-4acc-b301-e7aabfc6490a service cinder] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:11:12.296295 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1519946-71e8-4acc-b301-e7aabfc6490a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:11:12.298593 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-f1519946-71e8-4acc-b301-e7aabfc6490a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:11:12.383492 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1519946-71e8-4acc-b301-e7aabfc6490a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:11:12.384987 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1519946-71e8-4acc-b301-e7aabfc6490a service cinder] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:11:12.403692 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1248/2494] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:11:12 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:11:15.490290 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c771575-bbb8-4a5c-88d0-7e95e1986eb7 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:11:15.490788 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c771575-bbb8-4a5c-88d0-7e95e1986eb7 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:11:15.490907 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c771575-bbb8-4a5c-88d0-7e95e1986eb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:11:15.512179 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-7c771575-bbb8-4a5c-88d0-7e95e1986eb7 None None] MFA Rules not processed for user `5338bc8c594e4bc9bc41019226bf8d87`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:11:15.513444 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-7c771575-bbb8-4a5c-88d0-7e95e1986eb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:11:15.522488 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1247/2495] 10.222.0.64 () {58 vars in 967 bytes} [Tue Aug 2 10:11:15 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:11:15.546523 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1519946-71e8-4acc-b301-e7aabfc6490a service cinder] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:11:15.558945 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1519946-71e8-4acc-b301-e7aabfc6490a service cinder] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:11:15.564122 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c1c81e-07d6-440d-889a-1f9413daa7c8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a371e3df3764a4cbb1b8368aa74bd65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:11:15.564772 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c1c81e-07d6-440d-889a-1f9413daa7c8 service swift] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:11:15.565062 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c1c81e-07d6-440d-889a-1f9413daa7c8 service swift] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:11:15.565344 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c1c81e-07d6-440d-889a-1f9413daa7c8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:11:15.574232 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c1c81e-07d6-440d-889a-1f9413daa7c8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:11:15.575839 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c1c81e-07d6-440d-889a-1f9413daa7c8 service swift] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:11:15.584653 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1249/2496] 10.222.0.64 () {60 vars in 1384 bytes} [Tue Aug 2 10:11:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 02 10:11:25.506793 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=3 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:11:25.581193 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=47 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:11:44.995789 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c771575-bbb8-4a5c-88d0-7e95e1986eb7 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:11:45.024876 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c771575-bbb8-4a5c-88d0-7e95e1986eb7 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:11:45.034675 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de21f486-9510-4857-99f1-f81e579681c8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99655e1f9ca24472a86cd26b0dbd1fc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:11:45.035662 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de21f486-9510-4857-99f1-f81e579681c8 service nova] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:11:45.036150 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de21f486-9510-4857-99f1-f81e579681c8 service nova] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:11:45.036671 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de21f486-9510-4857-99f1-f81e579681c8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:11:45.039645 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-de21f486-9510-4857-99f1-f81e579681c8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:11:45.061392 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de21f486-9510-4857-99f1-f81e579681c8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:11:45.063727 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de21f486-9510-4857-99f1-f81e579681c8 service nova] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:11:45.082830 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1248/2497] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:11:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:11:54.102232 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c1c81e-07d6-440d-889a-1f9413daa7c8 service swift] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:11:54.114152 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c1c81e-07d6-440d-889a-1f9413daa7c8 service swift] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:11:54.119149 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b837f6-199f-4ed5-bec3-44380d4db7a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:11:54.119953 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b837f6-199f-4ed5-bec3-44380d4db7a1 service neutron] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:11:54.120374 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b837f6-199f-4ed5-bec3-44380d4db7a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:11:54.120756 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b837f6-199f-4ed5-bec3-44380d4db7a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:11:54.122653 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-21b837f6-199f-4ed5-bec3-44380d4db7a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:11:54.139268 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b837f6-199f-4ed5-bec3-44380d4db7a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:11:54.140788 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b837f6-199f-4ed5-bec3-44380d4db7a1 service neutron] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:11:54.154122 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1250/2498] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:11:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:11:55.076363 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=6 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:11:59.421252 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de21f486-9510-4857-99f1-f81e579681c8 service nova] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:11:59.423430 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-de21f486-9510-4857-99f1-f81e579681c8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:11:59.577284 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b837f6-199f-4ed5-bec3-44380d4db7a1 service neutron] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:11:59.582960 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-21b837f6-199f-4ed5-bec3-44380d4db7a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:11:59.610533 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de21f486-9510-4857-99f1-f81e579681c8 service nova] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:11:59.614053 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b837f6-199f-4ed5-bec3-44380d4db7a1 service neutron] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:11:59.624472 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e399336-06e5-4e2a-b31f-109d7e4f2565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:11:59.626123 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e399336-06e5-4e2a-b31f-109d7e4f2565 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:11:59.626567 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e399336-06e5-4e2a-b31f-109d7e4f2565 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:11:59.627565 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e399336-06e5-4e2a-b31f-109d7e4f2565 admin admin] PATH_INFO: `/v3/users/982a01f0d3a5459a9add3f6f23db5db7` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:11:59.630089 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e399336-06e5-4e2a-b31f-109d7e4f2565 admin admin] RBAC: Authorizing `identity:delete_user(user_id=982a01f0d3a5459a9add3f6f23db5db7)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:11:59.630946 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9882fb87-418c-4e9f-9633-743266c80a47 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b770f073c86b4d88837a8b1f796a8b1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:11:59.632286 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9882fb87-418c-4e9f-9633-743266c80a47 service placement] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:11:59.632898 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9882fb87-418c-4e9f-9633-743266c80a47 service placement] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:11:59.633809 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9882fb87-418c-4e9f-9633-743266c80a47 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:11:59.635075 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e399336-06e5-4e2a-b31f-109d7e4f2565 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:11:59.637216 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-9882fb87-418c-4e9f-9633-743266c80a47 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:11:59.652737 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9882fb87-418c-4e9f-9633-743266c80a47 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:11:59.654539 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9882fb87-418c-4e9f-9633-743266c80a47 service placement] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:11:59.666897 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1249/2499] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:11:59 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:11:59.716561 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-6e399336-06e5-4e2a-b31f-109d7e4f2565 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '982a01f0d3a5459a9add3f6f23db5db7'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:11:59.731052 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-6e399336-06e5-4e2a-b31f-109d7e4f2565 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '982a01f0d3a5459a9add3f6f23db5db7'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:11:59.743866 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-6e399336-06e5-4e2a-b31f-109d7e4f2565 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '982a01f0d3a5459a9add3f6f23db5db7'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:11:59.744896 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-6e399336-06e5-4e2a-b31f-109d7e4f2565 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '982a01f0d3a5459a9add3f6f23db5db7'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:11:59.745920 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-6e399336-06e5-4e2a-b31f-109d7e4f2565 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '982a01f0d3a5459a9add3f6f23db5db7'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:11:59.766312 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1251/2500] 10.222.0.64 () {58 vars in 1224 bytes} [Tue Aug 2 10:11:59 2022] DELETE /identity/v3/users/982a01f0d3a5459a9add3f6f23db5db7 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:11:59.982565 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9882fb87-418c-4e9f-9633-743266c80a47 service placement] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:11:59.999720 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9882fb87-418c-4e9f-9633-743266c80a47 service placement] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:12:00.084322 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba708cd0-6bb1-49c5-bc8f-ed31bc54a98b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:12:00.084869 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba708cd0-6bb1-49c5-bc8f-ed31bc54a98b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:00.085362 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba708cd0-6bb1-49c5-bc8f-ed31bc54a98b admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:00.085936 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba708cd0-6bb1-49c5-bc8f-ed31bc54a98b admin admin] PATH_INFO: `/v3/projects/dba5eb2dc9f549a5bf13a16b509dc3d4` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:00.087544 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba708cd0-6bb1-49c5-bc8f-ed31bc54a98b admin admin] RBAC: Authorizing `identity:delete_project(project_id=dba5eb2dc9f549a5bf13a16b509dc3d4)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:12:00.089619 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba708cd0-6bb1-49c5-bc8f-ed31bc54a98b admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:12:00.145568 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-ba708cd0-6bb1-49c5-bc8f-ed31bc54a98b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dba5eb2dc9f549a5bf13a16b509dc3d4', 'request_id': 'req-ba708cd0-6bb1-49c5-bc8f-ed31bc54a98b'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:12:00.152108 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-ba708cd0-6bb1-49c5-bc8f-ed31bc54a98b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dba5eb2dc9f549a5bf13a16b509dc3d4', 'request_id': 'req-ba708cd0-6bb1-49c5-bc8f-ed31bc54a98b'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:12:00.159925 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-ba708cd0-6bb1-49c5-bc8f-ed31bc54a98b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dba5eb2dc9f549a5bf13a16b509dc3d4', 'request_id': 'req-ba708cd0-6bb1-49c5-bc8f-ed31bc54a98b'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:12:00.180394 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-ba708cd0-6bb1-49c5-bc8f-ed31bc54a98b admin admin] The token cache is being invalidate because project dba5eb2dc9f549a5bf13a16b509dc3d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80154) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:12:00.181023 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-ba708cd0-6bb1-49c5-bc8f-ed31bc54a98b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:12:00.182123 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-ba708cd0-6bb1-49c5-bc8f-ed31bc54a98b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:12:00.185344 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1250/2501] 10.222.0.64 () {58 vars in 1233 bytes} [Tue Aug 2 10:11:59 2022] DELETE /identity/v3/projects/dba5eb2dc9f549a5bf13a16b509dc3d4 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:12:00.193947 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea79b6d-639f-450a-8e63-dad794673987 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:00.194885 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea79b6d-639f-450a-8e63-dad794673987 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:00.195077 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea79b6d-639f-450a-8e63-dad794673987 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:00.225527 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-eea79b6d-639f-450a-8e63-dad794673987 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:12:00.227251 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-eea79b6d-639f-450a-8e63-dad794673987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:12:00.331774 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1252/2502] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:12:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:12:00.345691 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770eda09-3d08-47e8-8a0a-131fbe9e468b None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:00.346208 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770eda09-3d08-47e8-8a0a-131fbe9e468b None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:00.346703 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770eda09-3d08-47e8-8a0a-131fbe9e468b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:00.377817 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-770eda09-3d08-47e8-8a0a-131fbe9e468b None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:12:00.379061 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-770eda09-3d08-47e8-8a0a-131fbe9e468b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:12:00.387757 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1251/2503] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:12:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:12:00.393505 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea79b6d-639f-450a-8e63-dad794673987 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:12:00.404698 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea79b6d-639f-450a-8e63-dad794673987 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:12:00.409741 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a826de9-5165-411a-a582-8d81d2d29b57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:12:00.410532 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a826de9-5165-411a-a582-8d81d2d29b57 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:00.410918 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a826de9-5165-411a-a582-8d81d2d29b57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:00.411291 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a826de9-5165-411a-a582-8d81d2d29b57 admin admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:00.412139 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a826de9-5165-411a-a582-8d81d2d29b57 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:12:00.412642 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a826de9-5165-411a-a582-8d81d2d29b57 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:12:00.414387 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a826de9-5165-411a-a582-8d81d2d29b57 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:12:00.428621 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1253/2504] 10.222.0.64 () {58 vars in 1153 bytes} [Tue Aug 2 10:12:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:12:00.434909 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770eda09-3d08-47e8-8a0a-131fbe9e468b None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:12:00.450206 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770eda09-3d08-47e8-8a0a-131fbe9e468b None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:12:00.457905 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2b82a0-5624-4519-a4a0-f3cc126d4dc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:12:00.458849 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2b82a0-5624-4519-a4a0-f3cc126d4dc9 admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:00.459289 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2b82a0-5624-4519-a4a0-f3cc126d4dc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:00.459712 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2b82a0-5624-4519-a4a0-f3cc126d4dc9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:00.460903 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2b82a0-5624-4519-a4a0-f3cc126d4dc9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:12:00.464099 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2b82a0-5624-4519-a4a0-f3cc126d4dc9 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:12:00.485164 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:12:00.513800 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1252/2505] 10.222.0.64 () {60 vars in 1153 bytes} [Tue Aug 2 10:12:00 2022] POST /identity/v3/projects => generated 403 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:12:00.523050 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a826de9-5165-411a-a582-8d81d2d29b57 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:12:00.539830 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a826de9-5165-411a-a582-8d81d2d29b57 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:12:00.596875 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:12:00.616583 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1e0b80-9f8f-43fe-8941-af949897f7ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:12:00.617210 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1e0b80-9f8f-43fe-8941-af949897f7ab admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:00.617491 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1e0b80-9f8f-43fe-8941-af949897f7ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:00.617765 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1e0b80-9f8f-43fe-8941-af949897f7ab admin admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:00.618437 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1e0b80-9f8f-43fe-8941-af949897f7ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:12:00.620402 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1e0b80-9f8f-43fe-8941-af949897f7ab admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:12:00.648796 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1254/2506] 10.222.0.64 () {60 vars in 1144 bytes} [Tue Aug 2 10:12:00 2022] POST /identity/v3/users => generated 302 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:12:00.654674 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2b82a0-5624-4519-a4a0-f3cc126d4dc9 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:12:00.669083 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2b82a0-5624-4519-a4a0-f3cc126d4dc9 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:12:00.676857 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadb7d4a-d5bf-4d07-b8fb-c456bb81a1d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:12:00.677830 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadb7d4a-d5bf-4d07-b8fb-c456bb81a1d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:00.678298 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadb7d4a-d5bf-4d07-b8fb-c456bb81a1d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:00.678759 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadb7d4a-d5bf-4d07-b8fb-c456bb81a1d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:00.679946 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadb7d4a-d5bf-4d07-b8fb-c456bb81a1d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:12:00.682818 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadb7d4a-d5bf-4d07-b8fb-c456bb81a1d4 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:12:00.704593 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1253/2507] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:12:00 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:12:00.706503 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1e0b80-9f8f-43fe-8941-af949897f7ab admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:12:00.721931 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1e0b80-9f8f-43fe-8941-af949897f7ab admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:12:00.732862 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8d445d-408d-4337-a89f-6f7018a192a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:12:00.733434 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8d445d-408d-4337-a89f-6f7018a192a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:00.733725 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8d445d-408d-4337-a89f-6f7018a192a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:00.734011 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8d445d-408d-4337-a89f-6f7018a192a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:00.734733 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8d445d-408d-4337-a89f-6f7018a192a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:12:00.736493 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8d445d-408d-4337-a89f-6f7018a192a0 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:12:00.749333 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1255/2508] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:12:00 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:12:00.754079 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadb7d4a-d5bf-4d07-b8fb-c456bb81a1d4 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:12:00.765625 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadb7d4a-d5bf-4d07-b8fb-c456bb81a1d4 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:12:00.770928 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a2ce12-bada-4348-ac50-92a3a5566503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:12:00.771753 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a2ce12-bada-4348-ac50-92a3a5566503 admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:00.772012 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a2ce12-bada-4348-ac50-92a3a5566503 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:00.772386 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a2ce12-bada-4348-ac50-92a3a5566503 admin admin] PATH_INFO: `/v3/projects/029e117ce209421e8031d0e552bf301b/users/51f7b8fe88c84f12ac47c8fc25dc1c15/roles/66c1074780954f8e87ed11a5c4e8c36a` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:00.800677 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a2ce12-bada-4348-ac50-92a3a5566503 admin admin] RBAC: Authorizing `identity:create_grant(project_id=029e117ce209421e8031d0e552bf301b, user_id=51f7b8fe88c84f12ac47c8fc25dc1c15, role_id=66c1074780954f8e87ed11a5c4e8c36a)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:12:00.805846 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a2ce12-bada-4348-ac50-92a3a5566503 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:12:00.828349 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1254/2509] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 10:12:00 2022] PUT /identity/v3/projects/029e117ce209421e8031d0e552bf301b/users/51f7b8fe88c84f12ac47c8fc25dc1c15/roles/66c1074780954f8e87ed11a5c4e8c36a => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:12:00.836883 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8044bcf5-9634-4f56-8106-b79ac738ae72 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:00.837376 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8044bcf5-9634-4f56-8106-b79ac738ae72 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:00.837837 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8044bcf5-9634-4f56-8106-b79ac738ae72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:00.869123 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-8044bcf5-9634-4f56-8106-b79ac738ae72 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:12:00.871172 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-8044bcf5-9634-4f56-8106-b79ac738ae72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:12:00.976878 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1256/2510] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:12:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:12:00.992024 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a2ce12-bada-4348-ac50-92a3a5566503 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:12:00.994675 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-44a2ce12-bada-4348-ac50-92a3a5566503 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:12:01.098692 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a2ce12-bada-4348-ac50-92a3a5566503 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:12:01.105011 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b308891-55f7-4e89-82e7-d5066b24198b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:12:01.106046 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b308891-55f7-4e89-82e7-d5066b24198b service neutron] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:01.106536 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b308891-55f7-4e89-82e7-d5066b24198b service neutron] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:01.107014 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b308891-55f7-4e89-82e7-d5066b24198b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:01.121946 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b308891-55f7-4e89-82e7-d5066b24198b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:12:01.124866 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b308891-55f7-4e89-82e7-d5066b24198b service neutron] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:12:01.144266 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1255/2511] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:12:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:12:01.998541 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25f20b2-2529-4d80-9c4c-b77deed82291 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:01.999188 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25f20b2-2529-4d80-9c4c-b77deed82291 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:01.999451 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25f20b2-2529-4d80-9c4c-b77deed82291 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:02.044281 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-d25f20b2-2529-4d80-9c4c-b77deed82291 None None] MFA Rules not processed for user `51f7b8fe88c84f12ac47c8fc25dc1c15`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:12:02.046395 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-d25f20b2-2529-4d80-9c4c-b77deed82291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:12:02.166743 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1257/2512] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:12:01 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:12:02.171972 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b308891-55f7-4e89-82e7-d5066b24198b service neutron] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:12:02.182598 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b308891-55f7-4e89-82e7-d5066b24198b service neutron] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:12:02.192044 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55dc5e4-cb03-4c10-92e4-2e4aa5c090c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:12:02.193420 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55dc5e4-cb03-4c10-92e4-2e4aa5c090c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:02.193895 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55dc5e4-cb03-4c10-92e4-2e4aa5c090c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:02.194378 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55dc5e4-cb03-4c10-92e4-2e4aa5c090c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:02.195598 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55dc5e4-cb03-4c10-92e4-2e4aa5c090c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:12:02.199352 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55dc5e4-cb03-4c10-92e4-2e4aa5c090c3 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:12:02.250985 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1256/2513] 10.222.0.64 () {60 vars in 1153 bytes} [Tue Aug 2 10:12:02 2022] POST /identity/v3/projects => generated 405 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:12:02.257805 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25f20b2-2529-4d80-9c4c-b77deed82291 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:12:02.275801 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25f20b2-2529-4d80-9c4c-b77deed82291 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:12:02.340644 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:12:02.352894 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126a7274-5c65-432f-8793-10b010de9b64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:12:02.353926 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126a7274-5c65-432f-8793-10b010de9b64 admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:02.354412 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126a7274-5c65-432f-8793-10b010de9b64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:02.354887 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126a7274-5c65-432f-8793-10b010de9b64 admin admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:02.356052 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126a7274-5c65-432f-8793-10b010de9b64 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:12:02.359641 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126a7274-5c65-432f-8793-10b010de9b64 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:12:02.389949 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1258/2514] 10.222.0.64 () {60 vars in 1144 bytes} [Tue Aug 2 10:12:02 2022] POST /identity/v3/users => generated 309 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:12:02.395948 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55dc5e4-cb03-4c10-92e4-2e4aa5c090c3 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:12:02.412069 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55dc5e4-cb03-4c10-92e4-2e4aa5c090c3 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:12:02.419550 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d9ee60-94df-47f8-8ae9-7528ab4f28f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:12:02.420505 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d9ee60-94df-47f8-8ae9-7528ab4f28f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:02.420947 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d9ee60-94df-47f8-8ae9-7528ab4f28f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:02.421374 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d9ee60-94df-47f8-8ae9-7528ab4f28f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:02.422490 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d9ee60-94df-47f8-8ae9-7528ab4f28f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:12:02.425651 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d9ee60-94df-47f8-8ae9-7528ab4f28f4 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:12:02.443629 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1257/2515] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:12:02 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:12:02.448564 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126a7274-5c65-432f-8793-10b010de9b64 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:12:02.464482 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126a7274-5c65-432f-8793-10b010de9b64 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:12:02.471705 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcf0e21-6e09-4b17-9e9a-676ca82225ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:12:02.472731 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcf0e21-6e09-4b17-9e9a-676ca82225ce admin admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:02.473174 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcf0e21-6e09-4b17-9e9a-676ca82225ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:02.473610 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcf0e21-6e09-4b17-9e9a-676ca82225ce admin admin] PATH_INFO: `/v3/projects/3c6812d3348d44548994924c6afaacfc/users/f7a87a4ec77b4636bfb419cd50947f09/roles/1000cf318fa14915b8091599eaead5de` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:02.497755 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcf0e21-6e09-4b17-9e9a-676ca82225ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c6812d3348d44548994924c6afaacfc, user_id=f7a87a4ec77b4636bfb419cd50947f09, role_id=1000cf318fa14915b8091599eaead5de)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:12:02.502568 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcf0e21-6e09-4b17-9e9a-676ca82225ce admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:12:02.523463 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1259/2516] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 10:12:02 2022] PUT /identity/v3/projects/3c6812d3348d44548994924c6afaacfc/users/f7a87a4ec77b4636bfb419cd50947f09/roles/1000cf318fa14915b8091599eaead5de => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:12:02.530124 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d9ee60-94df-47f8-8ae9-7528ab4f28f4 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:12:02.547662 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d9ee60-94df-47f8-8ae9-7528ab4f28f4 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:12:02.644234 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f938716d-4712-4d98-849d-170cc3bbc1e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:12:02.645187 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f938716d-4712-4d98-849d-170cc3bbc1e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:02.645670 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f938716d-4712-4d98-849d-170cc3bbc1e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:02.646146 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f938716d-4712-4d98-849d-170cc3bbc1e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:02.647393 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f938716d-4712-4d98-849d-170cc3bbc1e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:12:02.650528 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f938716d-4712-4d98-849d-170cc3bbc1e5 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:12:02.667302 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1258/2517] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:12:02 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:12:02.671869 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcf0e21-6e09-4b17-9e9a-676ca82225ce admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:12:02.683552 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcf0e21-6e09-4b17-9e9a-676ca82225ce admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:12:02.688670 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f124295f-1d51-4f50-b8a7-a3c947e15704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:12:02.689993 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f124295f-1d51-4f50-b8a7-a3c947e15704 admin admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:02.690372 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f124295f-1d51-4f50-b8a7-a3c947e15704 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:02.690769 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f124295f-1d51-4f50-b8a7-a3c947e15704 admin admin] PATH_INFO: `/v3/projects/3c6812d3348d44548994924c6afaacfc/users/f7a87a4ec77b4636bfb419cd50947f09/roles/1000cf318fa14915b8091599eaead5de` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:02.693234 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f124295f-1d51-4f50-b8a7-a3c947e15704 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c6812d3348d44548994924c6afaacfc, user_id=f7a87a4ec77b4636bfb419cd50947f09, role_id=1000cf318fa14915b8091599eaead5de)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:12:02.697001 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f124295f-1d51-4f50-b8a7-a3c947e15704 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:12:02.709780 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1260/2518] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 10:12:02 2022] PUT /identity/v3/projects/3c6812d3348d44548994924c6afaacfc/users/f7a87a4ec77b4636bfb419cd50947f09/roles/1000cf318fa14915b8091599eaead5de => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:12:03.579482 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9896728b-ebfe-42bf-a041-fa6c647f7a0c None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:03.580455 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9896728b-ebfe-42bf-a041-fa6c647f7a0c None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:03.580631 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9896728b-ebfe-42bf-a041-fa6c647f7a0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:03.620147 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-9896728b-ebfe-42bf-a041-fa6c647f7a0c None None] MFA Rules not processed for user `f7a87a4ec77b4636bfb419cd50947f09`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:12:03.622177 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-9896728b-ebfe-42bf-a041-fa6c647f7a0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:12:03.640677 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:12:03.755061 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1259/2519] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:12:03 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:12:05.299986 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f124295f-1d51-4f50-b8a7-a3c947e15704 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:12:05.396755 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f124295f-1d51-4f50-b8a7-a3c947e15704 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:12:05.402634 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f76f20d-f2cb-4853-80af-00c4ae1664d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:12:05.403791 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f76f20d-f2cb-4853-80af-00c4ae1664d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:05.404313 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f76f20d-f2cb-4853-80af-00c4ae1664d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:05.404818 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f76f20d-f2cb-4853-80af-00c4ae1664d5 admin admin] PATH_INFO: `/v3/users/51f7b8fe88c84f12ac47c8fc25dc1c15` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:05.406834 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f76f20d-f2cb-4853-80af-00c4ae1664d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=51f7b8fe88c84f12ac47c8fc25dc1c15)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:12:05.410726 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f76f20d-f2cb-4853-80af-00c4ae1664d5 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:12:05.487833 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-0f76f20d-f2cb-4853-80af-00c4ae1664d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51f7b8fe88c84f12ac47c8fc25dc1c15'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:12:05.499331 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-0f76f20d-f2cb-4853-80af-00c4ae1664d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51f7b8fe88c84f12ac47c8fc25dc1c15'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:12:05.509699 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-0f76f20d-f2cb-4853-80af-00c4ae1664d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51f7b8fe88c84f12ac47c8fc25dc1c15'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:12:05.510800 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-0f76f20d-f2cb-4853-80af-00c4ae1664d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51f7b8fe88c84f12ac47c8fc25dc1c15'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:12:05.511834 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-0f76f20d-f2cb-4853-80af-00c4ae1664d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51f7b8fe88c84f12ac47c8fc25dc1c15'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:12:05.539730 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1261/2520] 10.222.0.64 () {58 vars in 1224 bytes} [Tue Aug 2 10:12:05 2022] DELETE /identity/v3/users/51f7b8fe88c84f12ac47c8fc25dc1c15 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:12:05.733012 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9896728b-ebfe-42bf-a041-fa6c647f7a0c None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:12:05.750915 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9896728b-ebfe-42bf-a041-fa6c647f7a0c None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:12:05.833208 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5927e1f9-2500-49f9-a7d9-35461f54d789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:12:05.834330 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5927e1f9-2500-49f9-a7d9-35461f54d789 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:05.834847 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5927e1f9-2500-49f9-a7d9-35461f54d789 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:05.835357 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5927e1f9-2500-49f9-a7d9-35461f54d789 admin admin] PATH_INFO: `/v3/projects/029e117ce209421e8031d0e552bf301b` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:05.837472 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5927e1f9-2500-49f9-a7d9-35461f54d789 admin admin] RBAC: Authorizing `identity:delete_project(project_id=029e117ce209421e8031d0e552bf301b)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:12:05.840616 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5927e1f9-2500-49f9-a7d9-35461f54d789 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:12:05.915197 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-5927e1f9-2500-49f9-a7d9-35461f54d789 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '029e117ce209421e8031d0e552bf301b', 'request_id': 'req-5927e1f9-2500-49f9-a7d9-35461f54d789'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:12:05.923548 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-5927e1f9-2500-49f9-a7d9-35461f54d789 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '029e117ce209421e8031d0e552bf301b', 'request_id': 'req-5927e1f9-2500-49f9-a7d9-35461f54d789'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:12:05.930738 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-5927e1f9-2500-49f9-a7d9-35461f54d789 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '029e117ce209421e8031d0e552bf301b', 'request_id': 'req-5927e1f9-2500-49f9-a7d9-35461f54d789'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:12:05.948872 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-5927e1f9-2500-49f9-a7d9-35461f54d789 admin admin] The token cache is being invalidate because project 029e117ce209421e8031d0e552bf301b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80154) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:12:05.949326 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-5927e1f9-2500-49f9-a7d9-35461f54d789 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:12:05.949943 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-5927e1f9-2500-49f9-a7d9-35461f54d789 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:12:05.951690 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1260/2521] 10.222.0.64 () {58 vars in 1233 bytes} [Tue Aug 2 10:12:05 2022] DELETE /identity/v3/projects/029e117ce209421e8031d0e552bf301b => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:12:05.956678 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f76f20d-f2cb-4853-80af-00c4ae1664d5 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:12:05.959225 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-0f76f20d-f2cb-4853-80af-00c4ae1664d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:12:05.982945 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f76f20d-f2cb-4853-80af-00c4ae1664d5 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:12:06.021501 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=84,keystone:INSERT=6,keystone:DELETE=10 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:12:06.087263 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be99f98-a71e-44f0-abe7-b0c342131e43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:12:06.088352 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be99f98-a71e-44f0-abe7-b0c342131e43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:06.088848 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be99f98-a71e-44f0-abe7-b0c342131e43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:06.089332 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be99f98-a71e-44f0-abe7-b0c342131e43 admin admin] PATH_INFO: `/v3/users/f7a87a4ec77b4636bfb419cd50947f09` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:06.091362 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be99f98-a71e-44f0-abe7-b0c342131e43 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7a87a4ec77b4636bfb419cd50947f09)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:12:06.095265 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be99f98-a71e-44f0-abe7-b0c342131e43 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:12:06.182484 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-8be99f98-a71e-44f0-abe7-b0c342131e43 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7a87a4ec77b4636bfb419cd50947f09'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:12:06.188845 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-8be99f98-a71e-44f0-abe7-b0c342131e43 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7a87a4ec77b4636bfb419cd50947f09'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:12:06.201936 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-8be99f98-a71e-44f0-abe7-b0c342131e43 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7a87a4ec77b4636bfb419cd50947f09'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:12:06.202851 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-8be99f98-a71e-44f0-abe7-b0c342131e43 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7a87a4ec77b4636bfb419cd50947f09'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:12:06.203803 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-8be99f98-a71e-44f0-abe7-b0c342131e43 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7a87a4ec77b4636bfb419cd50947f09'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:12:06.232361 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1262/2522] 10.222.0.64 () {58 vars in 1224 bytes} [Tue Aug 2 10:12:05 2022] DELETE /identity/v3/users/f7a87a4ec77b4636bfb419cd50947f09 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:12:06.455970 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5927e1f9-2500-49f9-a7d9-35461f54d789 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:12:06.458463 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-5927e1f9-2500-49f9-a7d9-35461f54d789 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:12:06.474055 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5927e1f9-2500-49f9-a7d9-35461f54d789 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:12:06.528013 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:12:06.558958 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0a2e9d-0c9a-4b7b-8082-bf98e3e7f312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:12:06.559782 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0a2e9d-0c9a-4b7b-8082-bf98e3e7f312 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:06.560119 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0a2e9d-0c9a-4b7b-8082-bf98e3e7f312 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:06.560478 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0a2e9d-0c9a-4b7b-8082-bf98e3e7f312 admin admin] PATH_INFO: `/v3/projects/3c6812d3348d44548994924c6afaacfc` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:06.561757 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0a2e9d-0c9a-4b7b-8082-bf98e3e7f312 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c6812d3348d44548994924c6afaacfc)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:12:06.563985 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0a2e9d-0c9a-4b7b-8082-bf98e3e7f312 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:12:06.626084 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-5e0a2e9d-0c9a-4b7b-8082-bf98e3e7f312 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c6812d3348d44548994924c6afaacfc', 'request_id': 'req-5e0a2e9d-0c9a-4b7b-8082-bf98e3e7f312'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:12:06.633043 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-5e0a2e9d-0c9a-4b7b-8082-bf98e3e7f312 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c6812d3348d44548994924c6afaacfc', 'request_id': 'req-5e0a2e9d-0c9a-4b7b-8082-bf98e3e7f312'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:12:06.639698 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-5e0a2e9d-0c9a-4b7b-8082-bf98e3e7f312 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c6812d3348d44548994924c6afaacfc', 'request_id': 'req-5e0a2e9d-0c9a-4b7b-8082-bf98e3e7f312'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:12:06.658575 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-5e0a2e9d-0c9a-4b7b-8082-bf98e3e7f312 admin admin] The token cache is being invalidate because project 3c6812d3348d44548994924c6afaacfc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80154) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:12:06.659088 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-5e0a2e9d-0c9a-4b7b-8082-bf98e3e7f312 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:12:06.660138 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-5e0a2e9d-0c9a-4b7b-8082-bf98e3e7f312 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:12:06.663367 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1261/2523] 10.222.0.64 () {58 vars in 1233 bytes} [Tue Aug 2 10:12:06 2022] DELETE /identity/v3/projects/3c6812d3348d44548994924c6afaacfc => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:12:06.669405 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fc9d79-10c5-4ad3-89fa-9ae189cdf038 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:06.669923 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fc9d79-10c5-4ad3-89fa-9ae189cdf038 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:06.670033 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fc9d79-10c5-4ad3-89fa-9ae189cdf038 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:06.693494 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-e6fc9d79-10c5-4ad3-89fa-9ae189cdf038 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:12:06.694867 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-e6fc9d79-10c5-4ad3-89fa-9ae189cdf038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:12:06.765081 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1263/2524] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:12:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:12:06.773065 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294a4d62-f6e1-4ea6-99c1-a09df9f8f550 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:06.773703 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294a4d62-f6e1-4ea6-99c1-a09df9f8f550 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:06.773703 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294a4d62-f6e1-4ea6-99c1-a09df9f8f550 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:06.802489 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-294a4d62-f6e1-4ea6-99c1-a09df9f8f550 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:12:06.803734 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-294a4d62-f6e1-4ea6-99c1-a09df9f8f550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:12:06.812009 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1262/2525] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:12:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:12:06.817142 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fc9d79-10c5-4ad3-89fa-9ae189cdf038 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:12:06.829543 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fc9d79-10c5-4ad3-89fa-9ae189cdf038 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:12:06.834940 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e6032e-7ef3-4411-866d-6d05043045c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:12:06.835570 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e6032e-7ef3-4411-866d-6d05043045c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:06.835851 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e6032e-7ef3-4411-866d-6d05043045c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:06.836130 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e6032e-7ef3-4411-866d-6d05043045c8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:06.836867 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e6032e-7ef3-4411-866d-6d05043045c8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:12:06.837172 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e6032e-7ef3-4411-866d-6d05043045c8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:12:06.838915 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e6032e-7ef3-4411-866d-6d05043045c8 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:12:06.854615 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1264/2526] 10.222.0.64 () {58 vars in 1153 bytes} [Tue Aug 2 10:12:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:12:06.860464 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294a4d62-f6e1-4ea6-99c1-a09df9f8f550 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:12:06.870197 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294a4d62-f6e1-4ea6-99c1-a09df9f8f550 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:12:06.875186 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3bee65-939e-4dd2-9d28-3d7ec2e2edaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:12:06.875794 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3bee65-939e-4dd2-9d28-3d7ec2e2edaf admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:06.876079 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3bee65-939e-4dd2-9d28-3d7ec2e2edaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:06.876406 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3bee65-939e-4dd2-9d28-3d7ec2e2edaf admin admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:06.877124 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3bee65-939e-4dd2-9d28-3d7ec2e2edaf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:12:06.879040 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3bee65-939e-4dd2-9d28-3d7ec2e2edaf admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:12:06.910903 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1263/2527] 10.222.0.64 () {60 vars in 1153 bytes} [Tue Aug 2 10:12:06 2022] POST /identity/v3/projects => generated 381 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:12:06.915441 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e6032e-7ef3-4411-866d-6d05043045c8 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:12:06.926945 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e6032e-7ef3-4411-866d-6d05043045c8 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:12:06.960292 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:12:06.997245 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908a6066-8bb8-4e47-83a7-221acbbe356e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:12:06.998291 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908a6066-8bb8-4e47-83a7-221acbbe356e admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:06.998783 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908a6066-8bb8-4e47-83a7-221acbbe356e admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:06.999273 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908a6066-8bb8-4e47-83a7-221acbbe356e admin admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:07.000515 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908a6066-8bb8-4e47-83a7-221acbbe356e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:12:07.004356 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908a6066-8bb8-4e47-83a7-221acbbe356e admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:12:07.037291 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1265/2528] 10.222.0.64 () {60 vars in 1144 bytes} [Tue Aug 2 10:12:06 2022] POST /identity/v3/users => generated 291 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:12:07.041875 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3bee65-939e-4dd2-9d28-3d7ec2e2edaf admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:12:07.053642 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3bee65-939e-4dd2-9d28-3d7ec2e2edaf admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:12:07.058673 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdfef1b-82f4-4cca-8be5-63daef038423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:12:07.059350 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdfef1b-82f4-4cca-8be5-63daef038423 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:07.059665 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdfef1b-82f4-4cca-8be5-63daef038423 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:07.060020 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdfef1b-82f4-4cca-8be5-63daef038423 admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:07.060883 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdfef1b-82f4-4cca-8be5-63daef038423 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:12:07.062518 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdfef1b-82f4-4cca-8be5-63daef038423 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:12:07.074970 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1264/2529] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:12:07 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:12:07.081306 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908a6066-8bb8-4e47-83a7-221acbbe356e admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:12:07.092519 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908a6066-8bb8-4e47-83a7-221acbbe356e admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:12:07.099417 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e493bbfb-6811-431d-8819-94cc5ac5b134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:12:07.100508 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e493bbfb-6811-431d-8819-94cc5ac5b134 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:07.101059 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e493bbfb-6811-431d-8819-94cc5ac5b134 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:07.101545 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e493bbfb-6811-431d-8819-94cc5ac5b134 admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:07.102793 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e493bbfb-6811-431d-8819-94cc5ac5b134 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:12:07.105914 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e493bbfb-6811-431d-8819-94cc5ac5b134 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:12:07.126351 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1266/2530] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:12:07 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:12:07.132412 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdfef1b-82f4-4cca-8be5-63daef038423 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:12:07.143768 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdfef1b-82f4-4cca-8be5-63daef038423 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:12:07.149057 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3830b5fb-ea1e-487b-9f03-ba36f248613e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:12:07.149847 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3830b5fb-ea1e-487b-9f03-ba36f248613e admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:07.150167 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3830b5fb-ea1e-487b-9f03-ba36f248613e admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:07.150480 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3830b5fb-ea1e-487b-9f03-ba36f248613e admin admin] PATH_INFO: `/v3/projects/407e7a137500483cbe45d0d837913ec0/users/e3d0e9eb87774edfabe4eed9f159263c/roles/66c1074780954f8e87ed11a5c4e8c36a` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:07.165712 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3830b5fb-ea1e-487b-9f03-ba36f248613e admin admin] RBAC: Authorizing `identity:create_grant(project_id=407e7a137500483cbe45d0d837913ec0, user_id=e3d0e9eb87774edfabe4eed9f159263c, role_id=66c1074780954f8e87ed11a5c4e8c36a)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:12:07.168417 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3830b5fb-ea1e-487b-9f03-ba36f248613e admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:12:07.183030 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1265/2531] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 10:12:07 2022] PUT /identity/v3/projects/407e7a137500483cbe45d0d837913ec0/users/e3d0e9eb87774edfabe4eed9f159263c/roles/66c1074780954f8e87ed11a5c4e8c36a => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:12:07.194882 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1d754a-8ebc-45d1-813b-126bde6c1771 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:07.195403 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1d754a-8ebc-45d1-813b-126bde6c1771 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:07.195897 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1d754a-8ebc-45d1-813b-126bde6c1771 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:07.240047 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-de1d754a-8ebc-45d1-813b-126bde6c1771 None None] MFA Rules not processed for user `e3d0e9eb87774edfabe4eed9f159263c`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:12:07.241701 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-de1d754a-8ebc-45d1-813b-126bde6c1771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:12:07.316746 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1267/2532] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:12:07 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:12:07.323839 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3830b5fb-ea1e-487b-9f03-ba36f248613e admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:12:07.407633 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3830b5fb-ea1e-487b-9f03-ba36f248613e admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:12:07.414000 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee885fa0-511b-4b6c-a734-25edcf59d959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:12:07.414998 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee885fa0-511b-4b6c-a734-25edcf59d959 admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:07.415326 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee885fa0-511b-4b6c-a734-25edcf59d959 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:07.415656 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee885fa0-511b-4b6c-a734-25edcf59d959 admin admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:07.416463 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee885fa0-511b-4b6c-a734-25edcf59d959 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:12:07.418695 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee885fa0-511b-4b6c-a734-25edcf59d959 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:12:07.432589 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:12:07.468308 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1266/2533] 10.222.0.64 () {60 vars in 1153 bytes} [Tue Aug 2 10:12:07 2022] POST /identity/v3/projects => generated 379 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:12:07.471996 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1d754a-8ebc-45d1-813b-126bde6c1771 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:12:07.489342 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1d754a-8ebc-45d1-813b-126bde6c1771 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:12:07.585831 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e238221-7e66-4f96-bfc7-551b24105bd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:12:07.586913 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e238221-7e66-4f96-bfc7-551b24105bd6 admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:07.587418 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e238221-7e66-4f96-bfc7-551b24105bd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:07.587907 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e238221-7e66-4f96-bfc7-551b24105bd6 admin admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:07.589192 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e238221-7e66-4f96-bfc7-551b24105bd6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:12:07.593036 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e238221-7e66-4f96-bfc7-551b24105bd6 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:12:07.623770 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1268/2534] 10.222.0.64 () {60 vars in 1144 bytes} [Tue Aug 2 10:12:07 2022] POST /identity/v3/users => generated 296 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:12:07.629047 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee885fa0-511b-4b6c-a734-25edcf59d959 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:12:07.648174 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee885fa0-511b-4b6c-a734-25edcf59d959 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:12:07.654955 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a41889-b1cb-469b-a534-5db3e67a15e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:12:07.656010 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a41889-b1cb-469b-a534-5db3e67a15e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:07.656561 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a41889-b1cb-469b-a534-5db3e67a15e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:07.657063 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a41889-b1cb-469b-a534-5db3e67a15e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:07.658444 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a41889-b1cb-469b-a534-5db3e67a15e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:12:07.661529 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a41889-b1cb-469b-a534-5db3e67a15e4 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:12:07.681146 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1267/2535] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:12:07 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:12:07.687100 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e238221-7e66-4f96-bfc7-551b24105bd6 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:12:07.701998 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e238221-7e66-4f96-bfc7-551b24105bd6 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:12:07.708181 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3144ea4-5d58-4b84-bb0f-e403329e1c7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:12:07.708918 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3144ea4-5d58-4b84-bb0f-e403329e1c7e admin admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:07.709204 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3144ea4-5d58-4b84-bb0f-e403329e1c7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:07.709479 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3144ea4-5d58-4b84-bb0f-e403329e1c7e admin admin] PATH_INFO: `/v3/projects/f182221cf18548d38205f6b63efa37f5/users/1b214fe0a7f746888712887106d44dd4/roles/1000cf318fa14915b8091599eaead5de` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:07.728055 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3144ea4-5d58-4b84-bb0f-e403329e1c7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f182221cf18548d38205f6b63efa37f5, user_id=1b214fe0a7f746888712887106d44dd4, role_id=1000cf318fa14915b8091599eaead5de)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:12:07.731208 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3144ea4-5d58-4b84-bb0f-e403329e1c7e admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:12:07.748890 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1269/2536] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 10:12:07 2022] PUT /identity/v3/projects/f182221cf18548d38205f6b63efa37f5/users/1b214fe0a7f746888712887106d44dd4/roles/1000cf318fa14915b8091599eaead5de => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:12:07.754313 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a41889-b1cb-469b-a534-5db3e67a15e4 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:12:07.769814 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a41889-b1cb-469b-a534-5db3e67a15e4 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:12:07.858913 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbf4689-32c7-4d2d-981e-3280cb465535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:12:07.859584 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbf4689-32c7-4d2d-981e-3280cb465535 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:07.859744 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbf4689-32c7-4d2d-981e-3280cb465535 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:07.860041 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbf4689-32c7-4d2d-981e-3280cb465535 admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:07.860770 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbf4689-32c7-4d2d-981e-3280cb465535 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:12:07.862433 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbf4689-32c7-4d2d-981e-3280cb465535 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:12:07.873187 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1268/2537] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:12:07 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:12:07.879280 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3144ea4-5d58-4b84-bb0f-e403329e1c7e admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:12:07.896812 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3144ea4-5d58-4b84-bb0f-e403329e1c7e admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:12:07.904547 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e7d688-9b84-42a7-9a1b-d0b5bf6b2847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:12:07.905655 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e7d688-9b84-42a7-9a1b-d0b5bf6b2847 admin admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:07.906146 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e7d688-9b84-42a7-9a1b-d0b5bf6b2847 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:07.906635 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e7d688-9b84-42a7-9a1b-d0b5bf6b2847 admin admin] PATH_INFO: `/v3/projects/f182221cf18548d38205f6b63efa37f5/users/1b214fe0a7f746888712887106d44dd4/roles/1000cf318fa14915b8091599eaead5de` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:07.910728 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e7d688-9b84-42a7-9a1b-d0b5bf6b2847 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f182221cf18548d38205f6b63efa37f5, user_id=1b214fe0a7f746888712887106d44dd4, role_id=1000cf318fa14915b8091599eaead5de)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:12:07.914742 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e7d688-9b84-42a7-9a1b-d0b5bf6b2847 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:12:07.917235 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:12:07.933025 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1270/2538] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 10:12:07 2022] PUT /identity/v3/projects/f182221cf18548d38205f6b63efa37f5/users/1b214fe0a7f746888712887106d44dd4/roles/1000cf318fa14915b8091599eaead5de => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:12:07.947051 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfc2730-adec-44bd-8265-9941be397002 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:07.947520 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfc2730-adec-44bd-8265-9941be397002 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:07.947971 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfc2730-adec-44bd-8265-9941be397002 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:07.995862 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-2dfc2730-adec-44bd-8265-9941be397002 None None] MFA Rules not processed for user `1b214fe0a7f746888712887106d44dd4`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:12:07.997985 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-2dfc2730-adec-44bd-8265-9941be397002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:12:08.099563 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1269/2539] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:12:07 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:12:08.115037 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e7d688-9b84-42a7-9a1b-d0b5bf6b2847 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:12:08.116791 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-a2e7d688-9b84-42a7-9a1b-d0b5bf6b2847 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:12:08.211536 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e7d688-9b84-42a7-9a1b-d0b5bf6b2847 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:12:08.215615 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27986db6-997b-4e2e-8fae-095a1ee8d998 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d7de62b9ee4039bdc79627ac7fb499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:12:08.216235 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27986db6-997b-4e2e-8fae-095a1ee8d998 service cinder] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:08.216547 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27986db6-997b-4e2e-8fae-095a1ee8d998 service cinder] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:08.216821 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27986db6-997b-4e2e-8fae-095a1ee8d998 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:08.227776 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27986db6-997b-4e2e-8fae-095a1ee8d998 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:12:08.229492 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27986db6-997b-4e2e-8fae-095a1ee8d998 service cinder] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:12:08.239784 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1271/2540] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:12:08 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:12:08.346065 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfc2730-adec-44bd-8265-9941be397002 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:12:08.363356 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfc2730-adec-44bd-8265-9941be397002 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:12:08.369583 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c2edd3-f14e-401a-bc88-f657c001e890 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d7de62b9ee4039bdc79627ac7fb499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:12:08.370551 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c2edd3-f14e-401a-bc88-f657c001e890 service cinder] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:08.371041 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c2edd3-f14e-401a-bc88-f657c001e890 service cinder] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:08.371535 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c2edd3-f14e-401a-bc88-f657c001e890 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:08.473041 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c2edd3-f14e-401a-bc88-f657c001e890 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:12:08.475332 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c2edd3-f14e-401a-bc88-f657c001e890 service cinder] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:12:08.480759 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:12:08.492851 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1270/2541] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:12:08 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:12:13.826403 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3c4177-1a26-427b-a803-dba61ebd3780 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:13.827267 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3c4177-1a26-427b-a803-dba61ebd3780 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:13.827453 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3c4177-1a26-427b-a803-dba61ebd3780 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:13.857347 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-5a3c4177-1a26-427b-a803-dba61ebd3780 None None] MFA Rules not processed for user `c5cb1a83603e437ea576e127c8891fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:12:13.858601 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-5a3c4177-1a26-427b-a803-dba61ebd3780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:12:13.932875 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1272/2542] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:12:13 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:12:18.493282 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:12:23.923200 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:INSERT=1,keystone:SELECT=53 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:12:24.619067 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c2edd3-f14e-401a-bc88-f657c001e890 service cinder] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:12:24.700901 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c2edd3-f14e-401a-bc88-f657c001e890 service cinder] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:12:24.705672 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e38691-3686-4a1a-bfa5-733bea292eed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:12:24.706325 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e38691-3686-4a1a-bfa5-733bea292eed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:24.706603 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e38691-3686-4a1a-bfa5-733bea292eed admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:24.706882 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e38691-3686-4a1a-bfa5-733bea292eed admin admin] PATH_INFO: `/v3/users/e3d0e9eb87774edfabe4eed9f159263c` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:24.708145 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e38691-3686-4a1a-bfa5-733bea292eed admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3d0e9eb87774edfabe4eed9f159263c)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:12:24.710408 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e38691-3686-4a1a-bfa5-733bea292eed admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:12:24.770917 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-d5e38691-3686-4a1a-bfa5-733bea292eed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3d0e9eb87774edfabe4eed9f159263c'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:12:24.779734 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-d5e38691-3686-4a1a-bfa5-733bea292eed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3d0e9eb87774edfabe4eed9f159263c'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:12:24.780601 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-d5e38691-3686-4a1a-bfa5-733bea292eed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3d0e9eb87774edfabe4eed9f159263c'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:12:24.790159 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-d5e38691-3686-4a1a-bfa5-733bea292eed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3d0e9eb87774edfabe4eed9f159263c'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:12:24.806155 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-d5e38691-3686-4a1a-bfa5-733bea292eed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3d0e9eb87774edfabe4eed9f159263c'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:12:24.814044 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1271/2543] 10.222.0.64 () {58 vars in 1224 bytes} [Tue Aug 2 10:12:24 2022] DELETE /identity/v3/users/e3d0e9eb87774edfabe4eed9f159263c => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:12:24.819441 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6814bd1-a3e3-4a52-95c1-d956034777ef None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:24.819790 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6814bd1-a3e3-4a52-95c1-d956034777ef None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:24.820153 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6814bd1-a3e3-4a52-95c1-d956034777ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:24.844829 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-d6814bd1-a3e3-4a52-95c1-d956034777ef None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:12:24.847148 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-d6814bd1-a3e3-4a52-95c1-d956034777ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:12:24.938917 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1273/2544] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:12:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:12:24.951094 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e38691-3686-4a1a-bfa5-733bea292eed admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:12:24.953665 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-d5e38691-3686-4a1a-bfa5-733bea292eed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:12:25.024516 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e38691-3686-4a1a-bfa5-733bea292eed admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:12:25.030622 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e608ee2f-1ecb-4b00-80ed-9b75e8d8b393 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:12:25.031342 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e608ee2f-1ecb-4b00-80ed-9b75e8d8b393 service neutron] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:25.031757 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e608ee2f-1ecb-4b00-80ed-9b75e8d8b393 service neutron] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:25.032112 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e608ee2f-1ecb-4b00-80ed-9b75e8d8b393 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:25.045174 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e608ee2f-1ecb-4b00-80ed-9b75e8d8b393 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:12:25.047699 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e608ee2f-1ecb-4b00-80ed-9b75e8d8b393 service neutron] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:12:25.065412 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1272/2545] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:12:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:12:25.465610 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6814bd1-a3e3-4a52-95c1-d956034777ef None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:12:25.467245 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-d6814bd1-a3e3-4a52-95c1-d956034777ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:12:25.482047 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6814bd1-a3e3-4a52-95c1-d956034777ef None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:12:25.493244 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ea9b71-577a-4730-aa36-1200d5a088e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:12:25.494444 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ea9b71-577a-4730-aa36-1200d5a088e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:25.494987 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ea9b71-577a-4730-aa36-1200d5a088e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:25.495460 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ea9b71-577a-4730-aa36-1200d5a088e6 admin admin] PATH_INFO: `/v3/projects/407e7a137500483cbe45d0d837913ec0` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:25.497753 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ea9b71-577a-4730-aa36-1200d5a088e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=407e7a137500483cbe45d0d837913ec0)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:12:25.500004 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ea9b71-577a-4730-aa36-1200d5a088e6 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:12:25.606826 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-16ea9b71-577a-4730-aa36-1200d5a088e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '407e7a137500483cbe45d0d837913ec0', 'request_id': 'req-16ea9b71-577a-4730-aa36-1200d5a088e6'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:12:25.615273 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-16ea9b71-577a-4730-aa36-1200d5a088e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '407e7a137500483cbe45d0d837913ec0', 'request_id': 'req-16ea9b71-577a-4730-aa36-1200d5a088e6'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:12:25.627495 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-16ea9b71-577a-4730-aa36-1200d5a088e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '407e7a137500483cbe45d0d837913ec0', 'request_id': 'req-16ea9b71-577a-4730-aa36-1200d5a088e6'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:12:25.648497 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-16ea9b71-577a-4730-aa36-1200d5a088e6 admin admin] The token cache is being invalidate because project 407e7a137500483cbe45d0d837913ec0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80153) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:12:25.648899 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-16ea9b71-577a-4730-aa36-1200d5a088e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:12:25.650112 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-16ea9b71-577a-4730-aa36-1200d5a088e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:12:25.652860 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1274/2546] 10.222.0.64 () {58 vars in 1233 bytes} [Tue Aug 2 10:12:25 2022] DELETE /identity/v3/projects/407e7a137500483cbe45d0d837913ec0 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:12:25.658863 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e608ee2f-1ecb-4b00-80ed-9b75e8d8b393 service neutron] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:12:25.671350 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e608ee2f-1ecb-4b00-80ed-9b75e8d8b393 service neutron] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:12:25.712306 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:12:25.775707 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4ddd7a-e51f-4f61-a647-a97ff27bc8e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:12:25.776580 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4ddd7a-e51f-4f61-a647-a97ff27bc8e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:25.777117 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4ddd7a-e51f-4f61-a647-a97ff27bc8e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:25.777493 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4ddd7a-e51f-4f61-a647-a97ff27bc8e2 admin admin] PATH_INFO: `/v3/users/1b214fe0a7f746888712887106d44dd4` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:25.778834 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4ddd7a-e51f-4f61-a647-a97ff27bc8e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b214fe0a7f746888712887106d44dd4)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:12:25.781384 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4ddd7a-e51f-4f61-a647-a97ff27bc8e2 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:12:25.867941 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-4d4ddd7a-e51f-4f61-a647-a97ff27bc8e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b214fe0a7f746888712887106d44dd4'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:12:25.884072 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-4d4ddd7a-e51f-4f61-a647-a97ff27bc8e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b214fe0a7f746888712887106d44dd4'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:12:25.885812 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-4d4ddd7a-e51f-4f61-a647-a97ff27bc8e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b214fe0a7f746888712887106d44dd4'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:12:25.898468 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-4d4ddd7a-e51f-4f61-a647-a97ff27bc8e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b214fe0a7f746888712887106d44dd4'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:12:25.920502 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-4d4ddd7a-e51f-4f61-a647-a97ff27bc8e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b214fe0a7f746888712887106d44dd4'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:12:25.928828 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1273/2547] 10.222.0.64 () {58 vars in 1224 bytes} [Tue Aug 2 10:12:25 2022] DELETE /identity/v3/users/1b214fe0a7f746888712887106d44dd4 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:12:26.350408 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ea9b71-577a-4730-aa36-1200d5a088e6 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:12:26.351965 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-16ea9b71-577a-4730-aa36-1200d5a088e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:12:26.372460 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ea9b71-577a-4730-aa36-1200d5a088e6 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:12:26.464051 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbf7603-04b4-4e4c-a826-75d8559d8b09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:12:26.465271 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbf7603-04b4-4e4c-a826-75d8559d8b09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:26.465621 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbf7603-04b4-4e4c-a826-75d8559d8b09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:26.466063 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbf7603-04b4-4e4c-a826-75d8559d8b09 admin admin] PATH_INFO: `/v3/projects/f182221cf18548d38205f6b63efa37f5` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:26.467929 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbf7603-04b4-4e4c-a826-75d8559d8b09 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f182221cf18548d38205f6b63efa37f5)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:12:26.470565 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbf7603-04b4-4e4c-a826-75d8559d8b09 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:12:26.522034 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:12:26.553513 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-1dbf7603-04b4-4e4c-a826-75d8559d8b09 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f182221cf18548d38205f6b63efa37f5', 'request_id': 'req-1dbf7603-04b4-4e4c-a826-75d8559d8b09'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:12:26.563986 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-1dbf7603-04b4-4e4c-a826-75d8559d8b09 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f182221cf18548d38205f6b63efa37f5', 'request_id': 'req-1dbf7603-04b4-4e4c-a826-75d8559d8b09'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:12:26.573363 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-1dbf7603-04b4-4e4c-a826-75d8559d8b09 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f182221cf18548d38205f6b63efa37f5', 'request_id': 'req-1dbf7603-04b4-4e4c-a826-75d8559d8b09'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:12:26.589465 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-1dbf7603-04b4-4e4c-a826-75d8559d8b09 admin admin] The token cache is being invalidate because project f182221cf18548d38205f6b63efa37f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80153) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:12:26.589882 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-1dbf7603-04b4-4e4c-a826-75d8559d8b09 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:12:26.590667 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-1dbf7603-04b4-4e4c-a826-75d8559d8b09 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:12:26.592785 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1275/2548] 10.222.0.64 () {58 vars in 1233 bytes} [Tue Aug 2 10:12:26 2022] DELETE /identity/v3/projects/f182221cf18548d38205f6b63efa37f5 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:12:26.599661 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e279f3-6eb8-48a2-b5b4-38a2a9fb9930 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:26.599955 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e279f3-6eb8-48a2-b5b4-38a2a9fb9930 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:26.600232 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e279f3-6eb8-48a2-b5b4-38a2a9fb9930 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:26.625369 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-b7e279f3-6eb8-48a2-b5b4-38a2a9fb9930 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:12:26.628062 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-b7e279f3-6eb8-48a2-b5b4-38a2a9fb9930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:12:26.712431 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1274/2549] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:12:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:12:26.721308 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa63f86c-4c33-4674-8fa3-cc05395e4d5a None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:26.721529 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa63f86c-4c33-4674-8fa3-cc05395e4d5a None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:26.721953 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa63f86c-4c33-4674-8fa3-cc05395e4d5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:26.747549 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-aa63f86c-4c33-4674-8fa3-cc05395e4d5a None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:12:26.748893 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-aa63f86c-4c33-4674-8fa3-cc05395e4d5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:12:26.757917 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1276/2550] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:12:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:12:26.762600 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e279f3-6eb8-48a2-b5b4-38a2a9fb9930 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:12:26.775086 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e279f3-6eb8-48a2-b5b4-38a2a9fb9930 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:12:26.780081 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405d9204-a0c1-42fa-a6f8-1ad2480adf95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:12:26.780736 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405d9204-a0c1-42fa-a6f8-1ad2480adf95 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:26.781023 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405d9204-a0c1-42fa-a6f8-1ad2480adf95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:26.781328 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405d9204-a0c1-42fa-a6f8-1ad2480adf95 admin admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:26.782033 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405d9204-a0c1-42fa-a6f8-1ad2480adf95 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:12:26.782340 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405d9204-a0c1-42fa-a6f8-1ad2480adf95 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:12:26.784476 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405d9204-a0c1-42fa-a6f8-1ad2480adf95 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:12:26.797209 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1275/2551] 10.222.0.64 () {58 vars in 1153 bytes} [Tue Aug 2 10:12:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:12:26.802839 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa63f86c-4c33-4674-8fa3-cc05395e4d5a None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:12:26.812499 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa63f86c-4c33-4674-8fa3-cc05395e4d5a None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:12:26.820015 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500d3dbc-177c-46cf-bd5b-a0c208aebd9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:12:26.820966 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500d3dbc-177c-46cf-bd5b-a0c208aebd9a admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:26.821316 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500d3dbc-177c-46cf-bd5b-a0c208aebd9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:26.821752 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500d3dbc-177c-46cf-bd5b-a0c208aebd9a admin admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:26.822853 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500d3dbc-177c-46cf-bd5b-a0c208aebd9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:12:26.824930 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500d3dbc-177c-46cf-bd5b-a0c208aebd9a admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:12:26.864758 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1277/2552] 10.222.0.64 () {60 vars in 1153 bytes} [Tue Aug 2 10:12:26 2022] POST /identity/v3/projects => generated 355 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:12:26.870689 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405d9204-a0c1-42fa-a6f8-1ad2480adf95 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:12:26.884030 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405d9204-a0c1-42fa-a6f8-1ad2480adf95 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:12:26.947282 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:12:27.003326 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f91b99-1584-4d3c-b4d5-cb9058eb8e3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:12:27.004321 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f91b99-1584-4d3c-b4d5-cb9058eb8e3b admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:27.004699 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f91b99-1584-4d3c-b4d5-cb9058eb8e3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:27.005059 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f91b99-1584-4d3c-b4d5-cb9058eb8e3b admin admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:27.006341 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f91b99-1584-4d3c-b4d5-cb9058eb8e3b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:12:27.009523 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f91b99-1584-4d3c-b4d5-cb9058eb8e3b admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:12:27.047745 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1276/2553] 10.222.0.64 () {60 vars in 1144 bytes} [Tue Aug 2 10:12:26 2022] POST /identity/v3/users => generated 278 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:12:27.053429 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500d3dbc-177c-46cf-bd5b-a0c208aebd9a admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:12:27.072035 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500d3dbc-177c-46cf-bd5b-a0c208aebd9a admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:12:27.083025 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43a9904-57ac-4a89-accd-225968a123ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:12:27.084110 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43a9904-57ac-4a89-accd-225968a123ba admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:27.085173 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43a9904-57ac-4a89-accd-225968a123ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:27.086273 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43a9904-57ac-4a89-accd-225968a123ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:27.087628 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43a9904-57ac-4a89-accd-225968a123ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:12:27.090423 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43a9904-57ac-4a89-accd-225968a123ba admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:12:27.110594 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1278/2554] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:12:27 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:12:27.116509 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f91b99-1584-4d3c-b4d5-cb9058eb8e3b admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:12:27.132061 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f91b99-1584-4d3c-b4d5-cb9058eb8e3b admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:12:27.138565 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dde354-f628-4bc5-abb5-cdc8f1677092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:12:27.139415 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dde354-f628-4bc5-abb5-cdc8f1677092 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:27.139760 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dde354-f628-4bc5-abb5-cdc8f1677092 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:27.140072 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dde354-f628-4bc5-abb5-cdc8f1677092 admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:27.140942 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dde354-f628-4bc5-abb5-cdc8f1677092 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:12:27.142838 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dde354-f628-4bc5-abb5-cdc8f1677092 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:12:27.156638 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1277/2555] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:12:27 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:12:27.162796 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43a9904-57ac-4a89-accd-225968a123ba admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:12:27.178856 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43a9904-57ac-4a89-accd-225968a123ba admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:12:27.184595 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c003e3e0-62dd-4ac7-bcfb-e0e33c95a818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:12:27.185351 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c003e3e0-62dd-4ac7-bcfb-e0e33c95a818 admin admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:27.185676 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c003e3e0-62dd-4ac7-bcfb-e0e33c95a818 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:27.185996 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c003e3e0-62dd-4ac7-bcfb-e0e33c95a818 admin admin] PATH_INFO: `/v3/projects/3025f4dfa1b246bbbc737de611714c17/users/13ba07833c9a49fcbdc24655378b3aa7/roles/66c1074780954f8e87ed11a5c4e8c36a` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:27.214411 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c003e3e0-62dd-4ac7-bcfb-e0e33c95a818 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3025f4dfa1b246bbbc737de611714c17, user_id=13ba07833c9a49fcbdc24655378b3aa7, role_id=66c1074780954f8e87ed11a5c4e8c36a)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:12:27.218904 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c003e3e0-62dd-4ac7-bcfb-e0e33c95a818 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:12:27.242733 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1279/2556] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 10:12:27 2022] PUT /identity/v3/projects/3025f4dfa1b246bbbc737de611714c17/users/13ba07833c9a49fcbdc24655378b3aa7/roles/66c1074780954f8e87ed11a5c4e8c36a => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:12:27.254600 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b02801-6034-4551-bf58-889dfbd2ccb1 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:27.254854 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b02801-6034-4551-bf58-889dfbd2ccb1 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:27.255076 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b02801-6034-4551-bf58-889dfbd2ccb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:27.293300 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-10b02801-6034-4551-bf58-889dfbd2ccb1 None None] MFA Rules not processed for user `13ba07833c9a49fcbdc24655378b3aa7`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:12:27.294626 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-10b02801-6034-4551-bf58-889dfbd2ccb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:12:27.370705 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1278/2557] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:12:27 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:12:27.376892 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c003e3e0-62dd-4ac7-bcfb-e0e33c95a818 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:12:27.462398 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c003e3e0-62dd-4ac7-bcfb-e0e33c95a818 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:12:27.467344 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329c8bcf-49d4-465a-98fd-c315a5cc5697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:12:27.467979 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329c8bcf-49d4-465a-98fd-c315a5cc5697 admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:27.468312 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329c8bcf-49d4-465a-98fd-c315a5cc5697 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:27.468598 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329c8bcf-49d4-465a-98fd-c315a5cc5697 admin admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:27.469277 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329c8bcf-49d4-465a-98fd-c315a5cc5697 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:12:27.471291 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329c8bcf-49d4-465a-98fd-c315a5cc5697 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:12:27.508557 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1280/2558] 10.222.0.64 () {60 vars in 1153 bytes} [Tue Aug 2 10:12:27 2022] POST /identity/v3/projects => generated 357 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:12:27.515383 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b02801-6034-4551-bf58-889dfbd2ccb1 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:12:27.528449 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b02801-6034-4551-bf58-889dfbd2ccb1 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:12:27.607248 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29ad7a5-ee5a-4712-8eb2-060a8ce75a72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:12:27.608342 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29ad7a5-ee5a-4712-8eb2-060a8ce75a72 admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:27.608859 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29ad7a5-ee5a-4712-8eb2-060a8ce75a72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:27.609355 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29ad7a5-ee5a-4712-8eb2-060a8ce75a72 admin admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:27.610532 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29ad7a5-ee5a-4712-8eb2-060a8ce75a72 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:12:27.613777 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29ad7a5-ee5a-4712-8eb2-060a8ce75a72 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:12:27.644057 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1279/2559] 10.222.0.64 () {60 vars in 1144 bytes} [Tue Aug 2 10:12:27 2022] POST /identity/v3/users => generated 285 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:12:27.650086 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329c8bcf-49d4-465a-98fd-c315a5cc5697 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:12:27.662771 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329c8bcf-49d4-465a-98fd-c315a5cc5697 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:12:27.672981 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a287110-ee2f-4b81-8b40-e8afb64aff0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:12:27.673758 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a287110-ee2f-4b81-8b40-e8afb64aff0a admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:27.674180 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a287110-ee2f-4b81-8b40-e8afb64aff0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:27.674535 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a287110-ee2f-4b81-8b40-e8afb64aff0a admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:27.675527 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a287110-ee2f-4b81-8b40-e8afb64aff0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:12:27.677562 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a287110-ee2f-4b81-8b40-e8afb64aff0a admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:12:27.693463 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1281/2560] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:12:27 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:12:27.701712 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29ad7a5-ee5a-4712-8eb2-060a8ce75a72 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:12:27.713424 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29ad7a5-ee5a-4712-8eb2-060a8ce75a72 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:12:27.719394 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c6c142-4d97-48c5-8133-ac1938dd81d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:12:27.720137 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c6c142-4d97-48c5-8133-ac1938dd81d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:27.720455 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c6c142-4d97-48c5-8133-ac1938dd81d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:27.720781 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c6c142-4d97-48c5-8133-ac1938dd81d8 admin admin] PATH_INFO: `/v3/projects/afc33dc087984aa881889baf8227d908/users/0ad3a2b756994d128699055e53d7e18e/roles/1000cf318fa14915b8091599eaead5de` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:27.739439 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c6c142-4d97-48c5-8133-ac1938dd81d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=afc33dc087984aa881889baf8227d908, user_id=0ad3a2b756994d128699055e53d7e18e, role_id=1000cf318fa14915b8091599eaead5de)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:12:27.742269 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c6c142-4d97-48c5-8133-ac1938dd81d8 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:12:27.758838 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1280/2561] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 10:12:27 2022] PUT /identity/v3/projects/afc33dc087984aa881889baf8227d908/users/0ad3a2b756994d128699055e53d7e18e/roles/1000cf318fa14915b8091599eaead5de => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:12:27.770326 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a287110-ee2f-4b81-8b40-e8afb64aff0a admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:12:27.785864 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a287110-ee2f-4b81-8b40-e8afb64aff0a admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:12:27.824078 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:12:27.881230 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db247e8-7683-4a7a-a7ec-3be5274439a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:12:27.882360 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db247e8-7683-4a7a-a7ec-3be5274439a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:27.882873 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db247e8-7683-4a7a-a7ec-3be5274439a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:27.883328 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db247e8-7683-4a7a-a7ec-3be5274439a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:27.885171 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db247e8-7683-4a7a-a7ec-3be5274439a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:12:27.887419 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db247e8-7683-4a7a-a7ec-3be5274439a5 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:12:27.903388 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1282/2562] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:12:27 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:12:27.910973 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c6c142-4d97-48c5-8133-ac1938dd81d8 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:12:27.917299 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:12:27.926162 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c6c142-4d97-48c5-8133-ac1938dd81d8 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:12:27.932325 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d38c3b3-c96a-4720-bc7c-d6e78838feff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:12:27.933001 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d38c3b3-c96a-4720-bc7c-d6e78838feff admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:27.933293 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d38c3b3-c96a-4720-bc7c-d6e78838feff admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:27.933570 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d38c3b3-c96a-4720-bc7c-d6e78838feff admin admin] PATH_INFO: `/v3/projects/afc33dc087984aa881889baf8227d908/users/0ad3a2b756994d128699055e53d7e18e/roles/1000cf318fa14915b8091599eaead5de` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:27.935592 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d38c3b3-c96a-4720-bc7c-d6e78838feff admin admin] RBAC: Authorizing `identity:create_grant(project_id=afc33dc087984aa881889baf8227d908, user_id=0ad3a2b756994d128699055e53d7e18e, role_id=1000cf318fa14915b8091599eaead5de)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:12:27.938328 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d38c3b3-c96a-4720-bc7c-d6e78838feff admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:12:27.959932 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1281/2563] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 10:12:27 2022] PUT /identity/v3/projects/afc33dc087984aa881889baf8227d908/users/0ad3a2b756994d128699055e53d7e18e/roles/1000cf318fa14915b8091599eaead5de => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:12:27.978139 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6286e3a9-3e29-457c-a066-a4dd1d6b1e78 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:27.978605 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6286e3a9-3e29-457c-a066-a4dd1d6b1e78 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:27.979088 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6286e3a9-3e29-457c-a066-a4dd1d6b1e78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:28.012818 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-6286e3a9-3e29-457c-a066-a4dd1d6b1e78 None None] MFA Rules not processed for user `0ad3a2b756994d128699055e53d7e18e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:12:28.013981 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-6286e3a9-3e29-457c-a066-a4dd1d6b1e78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:12:28.088778 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1283/2564] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:12:27 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:12:28.107617 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d38c3b3-c96a-4720-bc7c-d6e78838feff admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:12:28.109424 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-1d38c3b3-c96a-4720-bc7c-d6e78838feff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:12:28.179240 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d38c3b3-c96a-4720-bc7c-d6e78838feff admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:12:28.185703 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac92a597-7a40-433f-8f28-779fbc1c2ac7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d7de62b9ee4039bdc79627ac7fb499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:12:28.186386 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac92a597-7a40-433f-8f28-779fbc1c2ac7 service cinder] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:28.186695 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac92a597-7a40-433f-8f28-779fbc1c2ac7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:28.186985 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac92a597-7a40-433f-8f28-779fbc1c2ac7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:28.199562 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac92a597-7a40-433f-8f28-779fbc1c2ac7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:12:28.201056 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac92a597-7a40-433f-8f28-779fbc1c2ac7 service cinder] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:12:28.212593 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1282/2565] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:12:28 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:12:28.302487 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6286e3a9-3e29-457c-a066-a4dd1d6b1e78 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:12:28.316139 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6286e3a9-3e29-457c-a066-a4dd1d6b1e78 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:12:28.328611 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa815c8-a7b4-42b4-9455-e7b39140f3d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d7de62b9ee4039bdc79627ac7fb499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:12:28.329830 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa815c8-a7b4-42b4-9455-e7b39140f3d7 service cinder] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:28.330315 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa815c8-a7b4-42b4-9455-e7b39140f3d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:28.330724 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa815c8-a7b4-42b4-9455-e7b39140f3d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:28.416945 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa815c8-a7b4-42b4-9455-e7b39140f3d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:12:28.418429 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa815c8-a7b4-42b4-9455-e7b39140f3d7 service cinder] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:12:28.433300 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1284/2566] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:12:28 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:12:31.074689 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac92a597-7a40-433f-8f28-779fbc1c2ac7 service cinder] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:12:31.076887 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-ac92a597-7a40-433f-8f28-779fbc1c2ac7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:12:31.211904 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac92a597-7a40-433f-8f28-779fbc1c2ac7 service cinder] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:12:31.224369 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dde091-29bf-4e03-a3ee-5fd1e508d18c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99655e1f9ca24472a86cd26b0dbd1fc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:12:31.225406 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dde091-29bf-4e03-a3ee-5fd1e508d18c service nova] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:31.226174 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dde091-29bf-4e03-a3ee-5fd1e508d18c service nova] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:31.227243 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dde091-29bf-4e03-a3ee-5fd1e508d18c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:31.231108 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-a6dde091-29bf-4e03-a3ee-5fd1e508d18c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:12:31.327453 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dde091-29bf-4e03-a3ee-5fd1e508d18c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:12:31.330236 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dde091-29bf-4e03-a3ee-5fd1e508d18c service nova] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:12:31.367461 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1283/2567] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:12:31 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:12:38.424889 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=80 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:12:41.360562 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=90,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:12:42.224971 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa815c8-a7b4-42b4-9455-e7b39140f3d7 service cinder] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:12:42.226501 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-6fa815c8-a7b4-42b4-9455-e7b39140f3d7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:12:42.334665 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa815c8-a7b4-42b4-9455-e7b39140f3d7 service cinder] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:12:42.341792 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5acf4a4-8ab9-4777-85ca-414e8c9439a8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b770f073c86b4d88837a8b1f796a8b1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:12:42.342766 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5acf4a4-8ab9-4777-85ca-414e8c9439a8 service placement] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:12:42.343221 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5acf4a4-8ab9-4777-85ca-414e8c9439a8 service placement] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:12:42.343754 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5acf4a4-8ab9-4777-85ca-414e8c9439a8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:12:42.348086 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-c5acf4a4-8ab9-4777-85ca-414e8c9439a8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:12:42.363183 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5acf4a4-8ab9-4777-85ca-414e8c9439a8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:12:42.365158 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5acf4a4-8ab9-4777-85ca-414e8c9439a8 service placement] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:12:42.386648 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1285/2568] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:12:42 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:12:52.374376 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=32 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:13:27.733771 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dde091-29bf-4e03-a3ee-5fd1e508d18c service nova] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:13:27.735519 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-a6dde091-29bf-4e03-a3ee-5fd1e508d18c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:13:27.803809 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dde091-29bf-4e03-a3ee-5fd1e508d18c service nova] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:13:27.807972 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ed3038-921f-4caa-bbbe-21c434047514 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:13:27.808693 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ed3038-921f-4caa-bbbe-21c434047514 service neutron] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:13:27.809017 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ed3038-921f-4caa-bbbe-21c434047514 service neutron] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:13:27.809335 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ed3038-921f-4caa-bbbe-21c434047514 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:13:27.811631 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-79ed3038-921f-4caa-bbbe-21c434047514 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:13:27.928755 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ed3038-921f-4caa-bbbe-21c434047514 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:13:27.930858 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ed3038-921f-4caa-bbbe-21c434047514 service neutron] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:13:27.952862 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1284/2569] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:13:27 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:13:37.949548 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=61 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:13:44.416576 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5acf4a4-8ab9-4777-85ca-414e8c9439a8 service placement] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:13:44.438937 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5acf4a4-8ab9-4777-85ca-414e8c9439a8 service placement] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:13:44.450155 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572d6f89-ecff-4de3-ba2f-9162f1bc8bc0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99655e1f9ca24472a86cd26b0dbd1fc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:13:44.451622 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572d6f89-ecff-4de3-ba2f-9162f1bc8bc0 service nova] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:13:44.452447 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572d6f89-ecff-4de3-ba2f-9162f1bc8bc0 service nova] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:13:44.453183 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572d6f89-ecff-4de3-ba2f-9162f1bc8bc0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:13:44.456932 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-572d6f89-ecff-4de3-ba2f-9162f1bc8bc0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:13:44.480957 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572d6f89-ecff-4de3-ba2f-9162f1bc8bc0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:13:44.484700 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572d6f89-ecff-4de3-ba2f-9162f1bc8bc0 service nova] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:13:44.520368 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1286/2570] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:13:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:13:54.499674 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=6 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:14:33.880005 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ed3038-921f-4caa-bbbe-21c434047514 service neutron] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:14:33.890415 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ed3038-921f-4caa-bbbe-21c434047514 service neutron] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:14:33.894804 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce169323-fb11-4ef9-a31c-707fdd1c2072 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:14:33.895592 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce169323-fb11-4ef9-a31c-707fdd1c2072 service neutron] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:14:33.895923 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce169323-fb11-4ef9-a31c-707fdd1c2072 service neutron] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:14:33.896282 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce169323-fb11-4ef9-a31c-707fdd1c2072 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:14:33.898784 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-ce169323-fb11-4ef9-a31c-707fdd1c2072 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:14:33.922216 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce169323-fb11-4ef9-a31c-707fdd1c2072 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:14:33.922216 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce169323-fb11-4ef9-a31c-707fdd1c2072 service neutron] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:14:33.941295 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1285/2571] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:14:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:14:43.931944 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=6 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:15:15.340676 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572d6f89-ecff-4de3-ba2f-9162f1bc8bc0 service nova] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:15:15.360124 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572d6f89-ecff-4de3-ba2f-9162f1bc8bc0 service nova] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:15:15.365243 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9044730-ac72-4a1e-953b-f1f86eef306a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:15:15.366101 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9044730-ac72-4a1e-953b-f1f86eef306a service neutron] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:15:15.366430 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9044730-ac72-4a1e-953b-f1f86eef306a service neutron] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:15:15.366749 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9044730-ac72-4a1e-953b-f1f86eef306a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:15:15.369227 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-c9044730-ac72-4a1e-953b-f1f86eef306a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:15:15.389836 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9044730-ac72-4a1e-953b-f1f86eef306a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:15:15.391311 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9044730-ac72-4a1e-953b-f1f86eef306a service neutron] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:15:15.404516 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1287/2572] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:15:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:15:25.398856 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=6 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:15:37.778032 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce169323-fb11-4ef9-a31c-707fdd1c2072 service neutron] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:15:37.779534 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-ce169323-fb11-4ef9-a31c-707fdd1c2072 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:15:37.796382 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce169323-fb11-4ef9-a31c-707fdd1c2072 service neutron] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:15:37.801998 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7bb63d-e7f7-4fe8-8e64-1c1fe4087a85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:15:37.802659 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7bb63d-e7f7-4fe8-8e64-1c1fe4087a85 service neutron] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:15:37.802983 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7bb63d-e7f7-4fe8-8e64-1c1fe4087a85 service neutron] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:15:37.803296 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7bb63d-e7f7-4fe8-8e64-1c1fe4087a85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:15:37.805713 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-fa7bb63d-e7f7-4fe8-8e64-1c1fe4087a85 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:15:37.820230 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7bb63d-e7f7-4fe8-8e64-1c1fe4087a85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:15:37.823203 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7bb63d-e7f7-4fe8-8e64-1c1fe4087a85 service neutron] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:15:37.837733 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1286/2573] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:15:37 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:15:44.715060 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9044730-ac72-4a1e-953b-f1f86eef306a service neutron] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:15:44.716531 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-c9044730-ac72-4a1e-953b-f1f86eef306a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:15:44.730689 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9044730-ac72-4a1e-953b-f1f86eef306a service neutron] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:15:44.736623 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0001ce72-54d8-4d0b-a2d2-93dbeeb18128 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d7de62b9ee4039bdc79627ac7fb499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:15:44.737246 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0001ce72-54d8-4d0b-a2d2-93dbeeb18128 service cinder] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:15:44.737536 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0001ce72-54d8-4d0b-a2d2-93dbeeb18128 service cinder] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:15:44.737816 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0001ce72-54d8-4d0b-a2d2-93dbeeb18128 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:15:44.739626 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-0001ce72-54d8-4d0b-a2d2-93dbeeb18128 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:15:44.814575 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0001ce72-54d8-4d0b-a2d2-93dbeeb18128 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:15:44.816059 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0001ce72-54d8-4d0b-a2d2-93dbeeb18128 service cinder] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:15:44.828020 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1288/2574] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:15:44 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:15:44.862879 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7bb63d-e7f7-4fe8-8e64-1c1fe4087a85 service neutron] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:15:44.873063 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7bb63d-e7f7-4fe8-8e64-1c1fe4087a85 service neutron] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:15:44.877010 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef96f11-e38e-4885-ab3c-da4db7c51ee9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d7de62b9ee4039bdc79627ac7fb499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:15:44.877554 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef96f11-e38e-4885-ab3c-da4db7c51ee9 service cinder] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:15:44.877838 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef96f11-e38e-4885-ab3c-da4db7c51ee9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:15:44.878110 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef96f11-e38e-4885-ab3c-da4db7c51ee9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:15:44.879969 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-9ef96f11-e38e-4885-ab3c-da4db7c51ee9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:15:44.953375 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef96f11-e38e-4885-ab3c-da4db7c51ee9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:15:44.955644 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef96f11-e38e-4885-ab3c-da4db7c51ee9 service cinder] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:15:44.965241 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1287/2575] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:15:44 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:15:54.821779 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=28 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:15:54.960763 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=34 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:17:31.554792 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0001ce72-54d8-4d0b-a2d2-93dbeeb18128 service cinder] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:31.573996 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0001ce72-54d8-4d0b-a2d2-93dbeeb18128 service cinder] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:31.579747 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114ea501-08f5-4202-8c91-9ae0720d29e1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99655e1f9ca24472a86cd26b0dbd1fc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:31.580395 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114ea501-08f5-4202-8c91-9ae0720d29e1 service nova] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:31.580708 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114ea501-08f5-4202-8c91-9ae0720d29e1 service nova] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:31.580963 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114ea501-08f5-4202-8c91-9ae0720d29e1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:31.589596 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114ea501-08f5-4202-8c91-9ae0720d29e1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:31.591196 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114ea501-08f5-4202-8c91-9ae0720d29e1 service nova] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:31.601129 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1289/2576] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:17:31 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:17:41.596736 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=6 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:17:45.224611 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef96f11-e38e-4885-ab3c-da4db7c51ee9 service cinder] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:45.257630 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef96f11-e38e-4885-ab3c-da4db7c51ee9 service cinder] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:45.263043 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b786ab96-03f0-477f-900a-e2e667a403b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d7de62b9ee4039bdc79627ac7fb499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:45.263887 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b786ab96-03f0-477f-900a-e2e667a403b9 service cinder] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:45.264408 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b786ab96-03f0-477f-900a-e2e667a403b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:45.264857 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b786ab96-03f0-477f-900a-e2e667a403b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:45.302294 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b786ab96-03f0-477f-900a-e2e667a403b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:45.304184 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b786ab96-03f0-477f-900a-e2e667a403b9 service cinder] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:45.315237 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1288/2577] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:17:45 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:17:48.168967 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114ea501-08f5-4202-8c91-9ae0720d29e1 service nova] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:17:48.272856 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114ea501-08f5-4202-8c91-9ae0720d29e1 service nova] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:17:48.282416 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da13b7ab-8973-4bce-9baf-9a48736a23ad service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b770f073c86b4d88837a8b1f796a8b1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:17:48.282757 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da13b7ab-8973-4bce-9baf-9a48736a23ad service placement] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:17:48.283251 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da13b7ab-8973-4bce-9baf-9a48736a23ad service placement] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:17:48.283722 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da13b7ab-8973-4bce-9baf-9a48736a23ad service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:17:48.298877 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da13b7ab-8973-4bce-9baf-9a48736a23ad service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:17:48.301114 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da13b7ab-8973-4bce-9baf-9a48736a23ad service placement] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:17:48.317738 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1290/2578] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:17:48 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:17:55.310472 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=17 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:17:58.308485 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=27 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:18:28.501109 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b786ab96-03f0-477f-900a-e2e667a403b9 service cinder] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:18:28.533808 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b786ab96-03f0-477f-900a-e2e667a403b9 service cinder] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:18:28.539390 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e489d1fc-50bf-46af-91ff-9e3141984e6b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99655e1f9ca24472a86cd26b0dbd1fc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:18:28.540006 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e489d1fc-50bf-46af-91ff-9e3141984e6b service nova] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:18:28.540323 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e489d1fc-50bf-46af-91ff-9e3141984e6b service nova] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:18:28.540604 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e489d1fc-50bf-46af-91ff-9e3141984e6b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:18:28.549224 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e489d1fc-50bf-46af-91ff-9e3141984e6b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:18:28.550813 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e489d1fc-50bf-46af-91ff-9e3141984e6b service nova] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:18:28.560136 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1289/2579] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:18:28 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:18:38.555833 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=13 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:19:46.167094 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da13b7ab-8973-4bce-9baf-9a48736a23ad service placement] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:19:46.186992 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da13b7ab-8973-4bce-9baf-9a48736a23ad service placement] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:19:46.190826 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56a216c-1611-48fc-9d64-6f316021f8b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d7de62b9ee4039bdc79627ac7fb499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:19:46.191435 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56a216c-1611-48fc-9d64-6f316021f8b9 service cinder] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:19:46.191722 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56a216c-1611-48fc-9d64-6f316021f8b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:19:46.191999 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56a216c-1611-48fc-9d64-6f316021f8b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:19:46.201993 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56a216c-1611-48fc-9d64-6f316021f8b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:19:46.204030 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56a216c-1611-48fc-9d64-6f316021f8b9 service cinder] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:19:46.213784 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1291/2580] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:19:46 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:19:46.241559 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e489d1fc-50bf-46af-91ff-9e3141984e6b service nova] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:19:46.252003 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e489d1fc-50bf-46af-91ff-9e3141984e6b service nova] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:19:46.255786 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cc04fc-15c4-45ae-bc33-7d4f4289abb4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d7de62b9ee4039bdc79627ac7fb499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:19:46.256435 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cc04fc-15c4-45ae-bc33-7d4f4289abb4 service cinder] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:19:46.256722 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cc04fc-15c4-45ae-bc33-7d4f4289abb4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:19:46.256998 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cc04fc-15c4-45ae-bc33-7d4f4289abb4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:19:46.268452 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cc04fc-15c4-45ae-bc33-7d4f4289abb4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:19:46.269895 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cc04fc-15c4-45ae-bc33-7d4f4289abb4 service cinder] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:19:46.281118 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1290/2581] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:19:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:19:56.209536 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=10 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:19:56.275918 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=6 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:20:40.745526 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56a216c-1611-48fc-9d64-6f316021f8b9 service cinder] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:20:40.765202 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56a216c-1611-48fc-9d64-6f316021f8b9 service cinder] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:20:40.768867 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729e6208-f6a1-4759-8726-698032ca22f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:20:40.769480 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729e6208-f6a1-4759-8726-698032ca22f6 service neutron] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:20:40.769765 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729e6208-f6a1-4759-8726-698032ca22f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:20:40.770040 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729e6208-f6a1-4759-8726-698032ca22f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:20:40.777835 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729e6208-f6a1-4759-8726-698032ca22f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:20:40.779311 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729e6208-f6a1-4759-8726-698032ca22f6 service neutron] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:20:40.788489 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1292/2582] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:20:40 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:20:45.993429 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cc04fc-15c4-45ae-bc33-7d4f4289abb4 service cinder] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:20:46.005040 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cc04fc-15c4-45ae-bc33-7d4f4289abb4 service cinder] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:20:46.010615 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9faee3a-a16d-474e-83ec-4ef7f2df1841 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99655e1f9ca24472a86cd26b0dbd1fc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:20:46.011230 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9faee3a-a16d-474e-83ec-4ef7f2df1841 service nova] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:20:46.011520 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9faee3a-a16d-474e-83ec-4ef7f2df1841 service nova] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:20:46.011799 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9faee3a-a16d-474e-83ec-4ef7f2df1841 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:20:46.019908 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9faee3a-a16d-474e-83ec-4ef7f2df1841 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:20:46.021550 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9faee3a-a16d-474e-83ec-4ef7f2df1841 service nova] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:20:46.049789 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1291/2583] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:20:45 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:20:50.784475 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=10 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:20:56.047925 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=13 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:21:46.148274 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729e6208-f6a1-4759-8726-698032ca22f6 service neutron] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:46.160683 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729e6208-f6a1-4759-8726-698032ca22f6 service neutron] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:46.164647 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f6f2c8-d0f2-4ac7-acd5-f3dced563667 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d7de62b9ee4039bdc79627ac7fb499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:46.165273 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f6f2c8-d0f2-4ac7-acd5-f3dced563667 service cinder] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:46.165573 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f6f2c8-d0f2-4ac7-acd5-f3dced563667 service cinder] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:46.165853 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f6f2c8-d0f2-4ac7-acd5-f3dced563667 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:46.175994 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f6f2c8-d0f2-4ac7-acd5-f3dced563667 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:46.178072 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f6f2c8-d0f2-4ac7-acd5-f3dced563667 service cinder] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:46.219314 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1293/2584] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:21:46 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:21:46.363350 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9faee3a-a16d-474e-83ec-4ef7f2df1841 service nova] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:21:46.373119 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9faee3a-a16d-474e-83ec-4ef7f2df1841 service nova] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:21:46.376754 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415f0bb3-34d5-473c-ac79-29c96204345a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d7de62b9ee4039bdc79627ac7fb499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:21:46.377376 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415f0bb3-34d5-473c-ac79-29c96204345a service cinder] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:21:46.377662 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415f0bb3-34d5-473c-ac79-29c96204345a service cinder] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:21:46.377946 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415f0bb3-34d5-473c-ac79-29c96204345a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:21:46.388995 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415f0bb3-34d5-473c-ac79-29c96204345a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:21:46.390446 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415f0bb3-34d5-473c-ac79-29c96204345a service cinder] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:21:46.405168 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1292/2585] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:21:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:21:56.214016 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=13 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:21:56.397121 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=6 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:22:54.095896 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f6f2c8-d0f2-4ac7-acd5-f3dced563667 service cinder] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:22:54.097349 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-67f6f2c8-d0f2-4ac7-acd5-f3dced563667 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:54.205419 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f6f2c8-d0f2-4ac7-acd5-f3dced563667 service cinder] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:22:54.210641 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61294ac7-f7eb-4302-b7bc-cc64856e6ae6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b770f073c86b4d88837a8b1f796a8b1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:22:54.211227 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61294ac7-f7eb-4302-b7bc-cc64856e6ae6 service placement] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:22:54.211508 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61294ac7-f7eb-4302-b7bc-cc64856e6ae6 service placement] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:22:54.211817 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61294ac7-f7eb-4302-b7bc-cc64856e6ae6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:22:54.213725 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-61294ac7-f7eb-4302-b7bc-cc64856e6ae6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:22:54.225964 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61294ac7-f7eb-4302-b7bc-cc64856e6ae6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:22:54.227583 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61294ac7-f7eb-4302-b7bc-cc64856e6ae6 service placement] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:22:54.237556 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1294/2586] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:22:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:23:04.233365 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=41 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:23:46.683105 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415f0bb3-34d5-473c-ac79-29c96204345a service cinder] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:46.710151 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61294ac7-f7eb-4302-b7bc-cc64856e6ae6 service placement] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:46.713226 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-61294ac7-f7eb-4302-b7bc-cc64856e6ae6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:46.755283 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415f0bb3-34d5-473c-ac79-29c96204345a service cinder] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:46.760870 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7258f10e-88cc-42fb-96ef-1408cb9ef34d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d7de62b9ee4039bdc79627ac7fb499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:46.761833 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7258f10e-88cc-42fb-96ef-1408cb9ef34d service cinder] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:46.762353 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7258f10e-88cc-42fb-96ef-1408cb9ef34d service cinder] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:46.763249 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7258f10e-88cc-42fb-96ef-1408cb9ef34d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:46.766107 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-7258f10e-88cc-42fb-96ef-1408cb9ef34d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:46.836653 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61294ac7-f7eb-4302-b7bc-cc64856e6ae6 service placement] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:46.841006 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33433ec7-64b6-4a00-8817-4783d7b8de15 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d7de62b9ee4039bdc79627ac7fb499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:46.841907 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33433ec7-64b6-4a00-8817-4783d7b8de15 service cinder] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:46.842242 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33433ec7-64b6-4a00-8817-4783d7b8de15 service cinder] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:46.842533 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33433ec7-64b6-4a00-8817-4783d7b8de15 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:46.845153 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-33433ec7-64b6-4a00-8817-4783d7b8de15 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:46.895549 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7258f10e-88cc-42fb-96ef-1408cb9ef34d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:46.897108 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7258f10e-88cc-42fb-96ef-1408cb9ef34d service cinder] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:46.924674 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1293/2587] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:23:46 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:23:46.950153 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33433ec7-64b6-4a00-8817-4783d7b8de15 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:46.951647 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33433ec7-64b6-4a00-8817-4783d7b8de15 service cinder] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:46.971286 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7258f10e-88cc-42fb-96ef-1408cb9ef34d service cinder] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:46.978815 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1295/2588] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:23:46 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:23:46.981788 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7258f10e-88cc-42fb-96ef-1408cb9ef34d service cinder] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:46.987370 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d7dfd1-1821-45f1-a86d-89c360748921 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d7de62b9ee4039bdc79627ac7fb499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:46.988399 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d7dfd1-1821-45f1-a86d-89c360748921 service cinder] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:46.988870 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d7dfd1-1821-45f1-a86d-89c360748921 service cinder] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:46.991481 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d7dfd1-1821-45f1-a86d-89c360748921 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:46.993883 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-24d7dfd1-1821-45f1-a86d-89c360748921 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:47.111015 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d7dfd1-1821-45f1-a86d-89c360748921 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:47.112525 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d7dfd1-1821-45f1-a86d-89c360748921 service cinder] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:47.122717 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33433ec7-64b6-4a00-8817-4783d7b8de15 service cinder] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:47.123596 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:23:47.133576 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33433ec7-64b6-4a00-8817-4783d7b8de15 service cinder] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:47.136895 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1294/2589] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:23:46 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:23:47.139570 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b74daf2-3e70-42ac-8502-5012cd354a11 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d7de62b9ee4039bdc79627ac7fb499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:47.140200 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b74daf2-3e70-42ac-8502-5012cd354a11 service cinder] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:47.140518 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b74daf2-3e70-42ac-8502-5012cd354a11 service cinder] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:47.140798 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b74daf2-3e70-42ac-8502-5012cd354a11 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:47.150273 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b74daf2-3e70-42ac-8502-5012cd354a11 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:47.151762 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b74daf2-3e70-42ac-8502-5012cd354a11 service cinder] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:47.162511 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1296/2590] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:23:47 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:23:47.463357 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d7dfd1-1821-45f1-a86d-89c360748921 service cinder] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:47.464868 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-24d7dfd1-1821-45f1-a86d-89c360748921 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:47.661159 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d7dfd1-1821-45f1-a86d-89c360748921 service cinder] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:47.666632 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ec10ed-9eeb-43f0-9846-620cb9ae9e37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:47.667440 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ec10ed-9eeb-43f0-9846-620cb9ae9e37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:47.667684 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ec10ed-9eeb-43f0-9846-620cb9ae9e37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:47.668171 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ec10ed-9eeb-43f0-9846-620cb9ae9e37 admin admin] PATH_INFO: `/v3/users/13ba07833c9a49fcbdc24655378b3aa7` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:47.669545 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ec10ed-9eeb-43f0-9846-620cb9ae9e37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=13ba07833c9a49fcbdc24655378b3aa7)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:47.672105 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ec10ed-9eeb-43f0-9846-620cb9ae9e37 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:47.730558 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-75ec10ed-9eeb-43f0-9846-620cb9ae9e37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13ba07833c9a49fcbdc24655378b3aa7'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:47.738577 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-75ec10ed-9eeb-43f0-9846-620cb9ae9e37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13ba07833c9a49fcbdc24655378b3aa7'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:47.739295 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-75ec10ed-9eeb-43f0-9846-620cb9ae9e37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13ba07833c9a49fcbdc24655378b3aa7'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:47.748134 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-75ec10ed-9eeb-43f0-9846-620cb9ae9e37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13ba07833c9a49fcbdc24655378b3aa7'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:47.763243 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-75ec10ed-9eeb-43f0-9846-620cb9ae9e37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13ba07833c9a49fcbdc24655378b3aa7'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:47.769550 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1295/2591] 10.222.0.64 () {58 vars in 1224 bytes} [Tue Aug 2 10:23:47 2022] DELETE /identity/v3/users/13ba07833c9a49fcbdc24655378b3aa7 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:23:47.774549 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30fc96e-c0eb-4313-a5a2-080571c5a7fb None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:47.775299 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30fc96e-c0eb-4313-a5a2-080571c5a7fb None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:47.775299 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30fc96e-c0eb-4313-a5a2-080571c5a7fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:47.805720 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-a30fc96e-c0eb-4313-a5a2-080571c5a7fb None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:23:47.806944 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-a30fc96e-c0eb-4313-a5a2-080571c5a7fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:47.858540 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:23:47.870509 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1297/2592] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:23:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:23:47.880491 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ec10ed-9eeb-43f0-9846-620cb9ae9e37 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:47.882062 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-75ec10ed-9eeb-43f0-9846-620cb9ae9e37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:47.941506 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:23:47.962214 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ec10ed-9eeb-43f0-9846-620cb9ae9e37 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:47.966068 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0cd3ad-f9c3-401e-ad8a-9826c469c0e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:47.966676 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0cd3ad-f9c3-401e-ad8a-9826c469c0e2 service neutron] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:47.966959 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0cd3ad-f9c3-401e-ad8a-9826c469c0e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:47.967744 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0cd3ad-f9c3-401e-ad8a-9826c469c0e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:47.977245 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0cd3ad-f9c3-401e-ad8a-9826c469c0e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:47.978683 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0cd3ad-f9c3-401e-ad8a-9826c469c0e2 service neutron] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:47.989104 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1296/2593] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:23:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:23:48.306916 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30fc96e-c0eb-4313-a5a2-080571c5a7fb None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:48.308637 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-a30fc96e-c0eb-4313-a5a2-080571c5a7fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:48.321756 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30fc96e-c0eb-4313-a5a2-080571c5a7fb None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:48.326760 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04fc16e-3ac7-446e-b07c-b595f751ac83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:48.328202 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04fc16e-3ac7-446e-b07c-b595f751ac83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:48.328531 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04fc16e-3ac7-446e-b07c-b595f751ac83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:48.328869 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04fc16e-3ac7-446e-b07c-b595f751ac83 admin admin] PATH_INFO: `/v3/projects/3025f4dfa1b246bbbc737de611714c17` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:48.330736 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04fc16e-3ac7-446e-b07c-b595f751ac83 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3025f4dfa1b246bbbc737de611714c17)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:48.332620 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04fc16e-3ac7-446e-b07c-b595f751ac83 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:48.393377 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-f04fc16e-3ac7-446e-b07c-b595f751ac83 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3025f4dfa1b246bbbc737de611714c17', 'request_id': 'req-f04fc16e-3ac7-446e-b07c-b595f751ac83'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:48.402936 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-f04fc16e-3ac7-446e-b07c-b595f751ac83 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3025f4dfa1b246bbbc737de611714c17', 'request_id': 'req-f04fc16e-3ac7-446e-b07c-b595f751ac83'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:48.413595 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-f04fc16e-3ac7-446e-b07c-b595f751ac83 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3025f4dfa1b246bbbc737de611714c17', 'request_id': 'req-f04fc16e-3ac7-446e-b07c-b595f751ac83'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:48.428624 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-f04fc16e-3ac7-446e-b07c-b595f751ac83 admin admin] The token cache is being invalidate because project 3025f4dfa1b246bbbc737de611714c17 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80153) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:23:48.428943 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-f04fc16e-3ac7-446e-b07c-b595f751ac83 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:48.429581 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-f04fc16e-3ac7-446e-b07c-b595f751ac83 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:48.431141 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1298/2594] 10.222.0.64 () {58 vars in 1233 bytes} [Tue Aug 2 10:23:48 2022] DELETE /identity/v3/projects/3025f4dfa1b246bbbc737de611714c17 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:23:48.435250 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0cd3ad-f9c3-401e-ad8a-9826c469c0e2 service neutron] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:48.445119 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0cd3ad-f9c3-401e-ad8a-9826c469c0e2 service neutron] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:48.507608 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc979d9c-a1e8-4d02-90f6-38b605298b67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:48.508278 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc979d9c-a1e8-4d02-90f6-38b605298b67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:48.508583 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc979d9c-a1e8-4d02-90f6-38b605298b67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:48.508863 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc979d9c-a1e8-4d02-90f6-38b605298b67 admin admin] PATH_INFO: `/v3/users/0ad3a2b756994d128699055e53d7e18e` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:48.522844 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc979d9c-a1e8-4d02-90f6-38b605298b67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ad3a2b756994d128699055e53d7e18e)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:48.524828 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc979d9c-a1e8-4d02-90f6-38b605298b67 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:48.579711 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-cc979d9c-a1e8-4d02-90f6-38b605298b67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ad3a2b756994d128699055e53d7e18e'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:48.586276 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-cc979d9c-a1e8-4d02-90f6-38b605298b67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ad3a2b756994d128699055e53d7e18e'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:48.587004 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-cc979d9c-a1e8-4d02-90f6-38b605298b67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ad3a2b756994d128699055e53d7e18e'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:48.598514 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-cc979d9c-a1e8-4d02-90f6-38b605298b67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ad3a2b756994d128699055e53d7e18e'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:48.617950 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-cc979d9c-a1e8-4d02-90f6-38b605298b67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ad3a2b756994d128699055e53d7e18e'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:48.626943 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1297/2595] 10.222.0.64 () {58 vars in 1224 bytes} [Tue Aug 2 10:23:48 2022] DELETE /identity/v3/users/0ad3a2b756994d128699055e53d7e18e => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:23:48.936472 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04fc16e-3ac7-446e-b07c-b595f751ac83 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:48.938006 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-f04fc16e-3ac7-446e-b07c-b595f751ac83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:48.955448 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04fc16e-3ac7-446e-b07c-b595f751ac83 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:49.031215 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbf73e7-9b62-4109-8407-3dccbda7c766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:49.031880 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbf73e7-9b62-4109-8407-3dccbda7c766 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:49.032168 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbf73e7-9b62-4109-8407-3dccbda7c766 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:49.032485 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbf73e7-9b62-4109-8407-3dccbda7c766 admin admin] PATH_INFO: `/v3/projects/afc33dc087984aa881889baf8227d908` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:49.044390 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbf73e7-9b62-4109-8407-3dccbda7c766 admin admin] RBAC: Authorizing `identity:delete_project(project_id=afc33dc087984aa881889baf8227d908)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:49.046772 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbf73e7-9b62-4109-8407-3dccbda7c766 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:49.108341 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-2bbf73e7-9b62-4109-8407-3dccbda7c766 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'afc33dc087984aa881889baf8227d908', 'request_id': 'req-2bbf73e7-9b62-4109-8407-3dccbda7c766'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:49.115930 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-2bbf73e7-9b62-4109-8407-3dccbda7c766 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'afc33dc087984aa881889baf8227d908', 'request_id': 'req-2bbf73e7-9b62-4109-8407-3dccbda7c766'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:49.126729 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-2bbf73e7-9b62-4109-8407-3dccbda7c766 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'afc33dc087984aa881889baf8227d908', 'request_id': 'req-2bbf73e7-9b62-4109-8407-3dccbda7c766'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:49.140513 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:23:49.149365 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-2bbf73e7-9b62-4109-8407-3dccbda7c766 admin admin] The token cache is being invalidate because project afc33dc087984aa881889baf8227d908 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80153) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:23:49.149670 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-2bbf73e7-9b62-4109-8407-3dccbda7c766 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:49.150627 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-2bbf73e7-9b62-4109-8407-3dccbda7c766 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:49.159704 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1299/2596] 10.222.0.64 () {58 vars in 1233 bytes} [Tue Aug 2 10:23:48 2022] DELETE /identity/v3/projects/afc33dc087984aa881889baf8227d908 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:23:49.161837 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac32761-1601-4efd-ac76-2ecb55f58dd3 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:49.162618 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac32761-1601-4efd-ac76-2ecb55f58dd3 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:49.162819 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac32761-1601-4efd-ac76-2ecb55f58dd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:49.223537 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-eac32761-1601-4efd-ac76-2ecb55f58dd3 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:23:49.224821 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-eac32761-1601-4efd-ac76-2ecb55f58dd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:49.243977 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:23:49.298449 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1298/2597] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:23:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:23:49.306808 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1fd2c1-034e-4f98-a5f0-088b5bc51707 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:49.307182 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1fd2c1-034e-4f98-a5f0-088b5bc51707 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:49.307544 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1fd2c1-034e-4f98-a5f0-088b5bc51707 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:49.332294 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-da1fd2c1-034e-4f98-a5f0-088b5bc51707 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:23:49.333664 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-da1fd2c1-034e-4f98-a5f0-088b5bc51707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:49.342972 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1300/2598] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:23:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:23:49.347475 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac32761-1601-4efd-ac76-2ecb55f58dd3 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:49.359085 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac32761-1601-4efd-ac76-2ecb55f58dd3 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:49.364628 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8db408-f0e7-42c2-8ce9-13e2e6b599d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:49.365325 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8db408-f0e7-42c2-8ce9-13e2e6b599d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:49.365662 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8db408-f0e7-42c2-8ce9-13e2e6b599d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:49.366017 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8db408-f0e7-42c2-8ce9-13e2e6b599d7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:49.367031 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8db408-f0e7-42c2-8ce9-13e2e6b599d7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:23:49.367386 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8db408-f0e7-42c2-8ce9-13e2e6b599d7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:49.369231 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8db408-f0e7-42c2-8ce9-13e2e6b599d7 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:49.381915 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1299/2599] 10.222.0.64 () {58 vars in 1153 bytes} [Tue Aug 2 10:23:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:23:49.386385 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1fd2c1-034e-4f98-a5f0-088b5bc51707 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:49.396295 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1fd2c1-034e-4f98-a5f0-088b5bc51707 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:49.401820 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50d611d-741d-45e0-a317-ce1203c8539e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:49.402445 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50d611d-741d-45e0-a317-ce1203c8539e admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:49.402732 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50d611d-741d-45e0-a317-ce1203c8539e admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:49.403016 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50d611d-741d-45e0-a317-ce1203c8539e admin admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:49.403705 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50d611d-741d-45e0-a317-ce1203c8539e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:49.406179 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50d611d-741d-45e0-a317-ce1203c8539e admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:49.468005 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1301/2600] 10.222.0.64 () {60 vars in 1153 bytes} [Tue Aug 2 10:23:49 2022] POST /identity/v3/projects => generated 379 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:23:49.472523 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8db408-f0e7-42c2-8ce9-13e2e6b599d7 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:49.483870 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8db408-f0e7-42c2-8ce9-13e2e6b599d7 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:49.547425 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b9cf5c-9ead-480e-836b-d787deeb48d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:49.548047 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b9cf5c-9ead-480e-836b-d787deeb48d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:49.548364 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b9cf5c-9ead-480e-836b-d787deeb48d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:49.548637 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b9cf5c-9ead-480e-836b-d787deeb48d7 admin admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:49.549319 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b9cf5c-9ead-480e-836b-d787deeb48d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:49.551245 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b9cf5c-9ead-480e-836b-d787deeb48d7 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:49.573987 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1300/2601] 10.222.0.64 () {60 vars in 1144 bytes} [Tue Aug 2 10:23:49 2022] POST /identity/v3/users => generated 290 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:23:49.578800 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50d611d-741d-45e0-a317-ce1203c8539e admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:49.589868 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50d611d-741d-45e0-a317-ce1203c8539e admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:49.595354 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef09764-aeb0-448c-bc86-446b9bc63f8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:49.595973 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef09764-aeb0-448c-bc86-446b9bc63f8b admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:49.596313 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef09764-aeb0-448c-bc86-446b9bc63f8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:49.596597 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef09764-aeb0-448c-bc86-446b9bc63f8b admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:49.597302 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef09764-aeb0-448c-bc86-446b9bc63f8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:49.598917 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef09764-aeb0-448c-bc86-446b9bc63f8b admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:49.609726 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1302/2602] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:23:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:23:49.614048 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b9cf5c-9ead-480e-836b-d787deeb48d7 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:49.625538 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b9cf5c-9ead-480e-836b-d787deeb48d7 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:49.630698 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636dc3bc-2fc3-4a8d-89dc-8859a9f796f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:49.631323 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636dc3bc-2fc3-4a8d-89dc-8859a9f796f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:49.631635 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636dc3bc-2fc3-4a8d-89dc-8859a9f796f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:49.632049 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636dc3bc-2fc3-4a8d-89dc-8859a9f796f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:49.632906 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636dc3bc-2fc3-4a8d-89dc-8859a9f796f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:49.635049 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636dc3bc-2fc3-4a8d-89dc-8859a9f796f2 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:49.652700 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1301/2603] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:23:49 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:23:49.657831 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef09764-aeb0-448c-bc86-446b9bc63f8b admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:49.668764 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef09764-aeb0-448c-bc86-446b9bc63f8b admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:49.674799 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6033da60-db51-4eac-9157-719041a91db9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:49.675668 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6033da60-db51-4eac-9157-719041a91db9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:49.676055 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6033da60-db51-4eac-9157-719041a91db9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:49.676459 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6033da60-db51-4eac-9157-719041a91db9 admin admin] PATH_INFO: `/v3/projects/f5418119f4564ac48a37f3821ced30f8/users/4a484ffd526543ee9838bf7467dc7595/roles/66c1074780954f8e87ed11a5c4e8c36a` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:49.695046 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6033da60-db51-4eac-9157-719041a91db9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5418119f4564ac48a37f3821ced30f8, user_id=4a484ffd526543ee9838bf7467dc7595, role_id=66c1074780954f8e87ed11a5c4e8c36a)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:49.698098 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6033da60-db51-4eac-9157-719041a91db9 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:49.715551 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1303/2604] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 10:23:49 2022] PUT /identity/v3/projects/f5418119f4564ac48a37f3821ced30f8/users/4a484ffd526543ee9838bf7467dc7595/roles/66c1074780954f8e87ed11a5c4e8c36a => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:23:49.728193 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ee3bd2-26a2-4e91-994d-22ffb627f6d1 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:49.728561 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ee3bd2-26a2-4e91-994d-22ffb627f6d1 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:49.728804 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ee3bd2-26a2-4e91-994d-22ffb627f6d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:49.762186 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-50ee3bd2-26a2-4e91-994d-22ffb627f6d1 None None] MFA Rules not processed for user `4a484ffd526543ee9838bf7467dc7595`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:23:49.763514 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-50ee3bd2-26a2-4e91-994d-22ffb627f6d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:49.961019 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1302/2605] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:23:49 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:23:49.965565 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6033da60-db51-4eac-9157-719041a91db9 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:50.161156 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6033da60-db51-4eac-9157-719041a91db9 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:50.166878 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee79b723-47e5-40a0-a601-e609fdd7fb67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:50.167854 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee79b723-47e5-40a0-a601-e609fdd7fb67 admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:50.168553 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee79b723-47e5-40a0-a601-e609fdd7fb67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:50.168988 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee79b723-47e5-40a0-a601-e609fdd7fb67 admin admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:50.169936 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee79b723-47e5-40a0-a601-e609fdd7fb67 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:50.172840 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee79b723-47e5-40a0-a601-e609fdd7fb67 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:50.209033 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1304/2606] 10.222.0.64 () {60 vars in 1153 bytes} [Tue Aug 2 10:23:49 2022] POST /identity/v3/projects => generated 379 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:23:50.213285 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ee3bd2-26a2-4e91-994d-22ffb627f6d1 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:50.225215 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ee3bd2-26a2-4e91-994d-22ffb627f6d1 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:50.237973 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:23:50.288932 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bba0955-082e-454f-97ed-dd9c047f5d75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:50.289544 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bba0955-082e-454f-97ed-dd9c047f5d75 admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:50.289831 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bba0955-082e-454f-97ed-dd9c047f5d75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:50.290279 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bba0955-082e-454f-97ed-dd9c047f5d75 admin admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:50.290810 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bba0955-082e-454f-97ed-dd9c047f5d75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:50.292792 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bba0955-082e-454f-97ed-dd9c047f5d75 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:50.326970 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1303/2607] 10.222.0.64 () {60 vars in 1144 bytes} [Tue Aug 2 10:23:50 2022] POST /identity/v3/users => generated 296 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:23:50.328487 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee79b723-47e5-40a0-a601-e609fdd7fb67 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:50.339383 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee79b723-47e5-40a0-a601-e609fdd7fb67 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:50.344969 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af6d51f-2363-48e7-8282-955c8fdd1987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:50.345584 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af6d51f-2363-48e7-8282-955c8fdd1987 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:50.345866 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af6d51f-2363-48e7-8282-955c8fdd1987 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:50.346138 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af6d51f-2363-48e7-8282-955c8fdd1987 admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:50.346832 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af6d51f-2363-48e7-8282-955c8fdd1987 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:50.348494 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af6d51f-2363-48e7-8282-955c8fdd1987 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:50.359297 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1305/2608] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:23:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:23:50.363370 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bba0955-082e-454f-97ed-dd9c047f5d75 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:50.374947 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bba0955-082e-454f-97ed-dd9c047f5d75 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:50.379674 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e847d016-9069-4908-81a1-5bbca8321fd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:50.380534 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e847d016-9069-4908-81a1-5bbca8321fd8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:50.380969 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e847d016-9069-4908-81a1-5bbca8321fd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:50.381248 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e847d016-9069-4908-81a1-5bbca8321fd8 admin admin] PATH_INFO: `/v3/projects/624aeca4fac34491a077be3a70494b9d/users/c23b9651744c47139511d4d83114c044/roles/1000cf318fa14915b8091599eaead5de` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:50.402093 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e847d016-9069-4908-81a1-5bbca8321fd8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=624aeca4fac34491a077be3a70494b9d, user_id=c23b9651744c47139511d4d83114c044, role_id=1000cf318fa14915b8091599eaead5de)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:50.406140 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e847d016-9069-4908-81a1-5bbca8321fd8 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:50.423075 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1304/2609] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 10:23:50 2022] PUT /identity/v3/projects/624aeca4fac34491a077be3a70494b9d/users/c23b9651744c47139511d4d83114c044/roles/1000cf318fa14915b8091599eaead5de => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:23:50.427102 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af6d51f-2363-48e7-8282-955c8fdd1987 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:50.437852 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af6d51f-2363-48e7-8282-955c8fdd1987 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:50.497810 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a75539c-0525-4697-bdcd-2b8cc78c2026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:50.499022 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a75539c-0525-4697-bdcd-2b8cc78c2026 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:50.499302 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a75539c-0525-4697-bdcd-2b8cc78c2026 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:50.499664 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a75539c-0525-4697-bdcd-2b8cc78c2026 admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:50.500407 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a75539c-0525-4697-bdcd-2b8cc78c2026 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:50.501999 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a75539c-0525-4697-bdcd-2b8cc78c2026 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:50.504417 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:23:50.515346 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1306/2610] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:23:50 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:23:50.518588 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e847d016-9069-4908-81a1-5bbca8321fd8 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:50.529055 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e847d016-9069-4908-81a1-5bbca8321fd8 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:50.533807 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53724523-0ab1-4661-9ea2-c9e2b53632c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:50.534471 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53724523-0ab1-4661-9ea2-c9e2b53632c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:50.534755 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53724523-0ab1-4661-9ea2-c9e2b53632c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:50.535028 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53724523-0ab1-4661-9ea2-c9e2b53632c5 admin admin] PATH_INFO: `/v3/projects/624aeca4fac34491a077be3a70494b9d/users/c23b9651744c47139511d4d83114c044/roles/1000cf318fa14915b8091599eaead5de` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:50.537004 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53724523-0ab1-4661-9ea2-c9e2b53632c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=624aeca4fac34491a077be3a70494b9d, user_id=c23b9651744c47139511d4d83114c044, role_id=1000cf318fa14915b8091599eaead5de)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:50.539611 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53724523-0ab1-4661-9ea2-c9e2b53632c5 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:50.550341 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1305/2611] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 10:23:50 2022] PUT /identity/v3/projects/624aeca4fac34491a077be3a70494b9d/users/c23b9651744c47139511d4d83114c044/roles/1000cf318fa14915b8091599eaead5de => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:23:50.620810 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d841f4-c54b-4e4d-b489-5f35c7dd0bf4 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:50.621110 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d841f4-c54b-4e4d-b489-5f35c7dd0bf4 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:50.621349 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d841f4-c54b-4e4d-b489-5f35c7dd0bf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:50.650716 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-c0d841f4-c54b-4e4d-b489-5f35c7dd0bf4 None None] MFA Rules not processed for user `c23b9651744c47139511d4d83114c044`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:23:50.651926 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-c0d841f4-c54b-4e4d-b489-5f35c7dd0bf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:50.726091 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1307/2612] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:23:50 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:23:50.737922 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53724523-0ab1-4661-9ea2-c9e2b53632c5 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:50.740014 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-53724523-0ab1-4661-9ea2-c9e2b53632c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:50.803902 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53724523-0ab1-4661-9ea2-c9e2b53632c5 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:50.808290 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5152b6bc-d2d0-41e7-b11c-65a9e3443c2b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d7de62b9ee4039bdc79627ac7fb499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:50.808920 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5152b6bc-d2d0-41e7-b11c-65a9e3443c2b service cinder] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:50.809207 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5152b6bc-d2d0-41e7-b11c-65a9e3443c2b service cinder] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:50.809485 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5152b6bc-d2d0-41e7-b11c-65a9e3443c2b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:50.819382 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5152b6bc-d2d0-41e7-b11c-65a9e3443c2b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:50.821126 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5152b6bc-d2d0-41e7-b11c-65a9e3443c2b service cinder] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:50.830736 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1306/2613] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:23:50 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:23:51.293551 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d841f4-c54b-4e4d-b489-5f35c7dd0bf4 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:51.373967 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d841f4-c54b-4e4d-b489-5f35c7dd0bf4 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:51.379655 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f923aaf-8513-4d17-b2aa-6a0f43d6d90b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:51.380450 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f923aaf-8513-4d17-b2aa-6a0f43d6d90b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:51.380957 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f923aaf-8513-4d17-b2aa-6a0f43d6d90b admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:51.380957 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f923aaf-8513-4d17-b2aa-6a0f43d6d90b admin admin] PATH_INFO: `/v3/users/4a484ffd526543ee9838bf7467dc7595` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:51.385728 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f923aaf-8513-4d17-b2aa-6a0f43d6d90b admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a484ffd526543ee9838bf7467dc7595)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:51.385728 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f923aaf-8513-4d17-b2aa-6a0f43d6d90b admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:51.440864 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-5f923aaf-8513-4d17-b2aa-6a0f43d6d90b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a484ffd526543ee9838bf7467dc7595'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:51.451013 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-5f923aaf-8513-4d17-b2aa-6a0f43d6d90b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a484ffd526543ee9838bf7467dc7595'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:51.460054 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-5f923aaf-8513-4d17-b2aa-6a0f43d6d90b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a484ffd526543ee9838bf7467dc7595'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:51.461362 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-5f923aaf-8513-4d17-b2aa-6a0f43d6d90b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a484ffd526543ee9838bf7467dc7595'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:51.461784 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-5f923aaf-8513-4d17-b2aa-6a0f43d6d90b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a484ffd526543ee9838bf7467dc7595'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:51.480609 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1308/2614] 10.222.0.64 () {58 vars in 1224 bytes} [Tue Aug 2 10:23:51 2022] DELETE /identity/v3/users/4a484ffd526543ee9838bf7467dc7595 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:23:51.485457 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca8e357-4ac8-4220-a704-08d2fdf67fea None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:51.485950 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca8e357-4ac8-4220-a704-08d2fdf67fea None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:51.486185 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca8e357-4ac8-4220-a704-08d2fdf67fea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:51.506906 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-8ca8e357-4ac8-4220-a704-08d2fdf67fea None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:23:51.508107 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-8ca8e357-4ac8-4220-a704-08d2fdf67fea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:51.568950 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1307/2615] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:23:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:23:51.580824 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f923aaf-8513-4d17-b2aa-6a0f43d6d90b admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:51.582266 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-5f923aaf-8513-4d17-b2aa-6a0f43d6d90b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:51.588321 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:23:51.643411 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f923aaf-8513-4d17-b2aa-6a0f43d6d90b admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:51.647116 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2483f22-e204-41ea-8074-78943c6b395d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:51.647731 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2483f22-e204-41ea-8074-78943c6b395d service neutron] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:51.648017 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2483f22-e204-41ea-8074-78943c6b395d service neutron] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:51.648321 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2483f22-e204-41ea-8074-78943c6b395d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:51.657363 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2483f22-e204-41ea-8074-78943c6b395d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:51.658799 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2483f22-e204-41ea-8074-78943c6b395d service neutron] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:51.671369 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1309/2616] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:23:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:23:51.954607 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca8e357-4ac8-4220-a704-08d2fdf67fea None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:51.956231 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-8ca8e357-4ac8-4220-a704-08d2fdf67fea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:51.969058 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca8e357-4ac8-4220-a704-08d2fdf67fea None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:51.975890 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83c0090-8328-4ad0-a347-d89681f0356b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:51.976902 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83c0090-8328-4ad0-a347-d89681f0356b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:51.977042 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83c0090-8328-4ad0-a347-d89681f0356b admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:51.977204 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83c0090-8328-4ad0-a347-d89681f0356b admin admin] PATH_INFO: `/v3/projects/f5418119f4564ac48a37f3821ced30f8` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:51.978460 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83c0090-8328-4ad0-a347-d89681f0356b admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5418119f4564ac48a37f3821ced30f8)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:51.980406 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83c0090-8328-4ad0-a347-d89681f0356b admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:51.989745 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:23:52.032682 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-e83c0090-8328-4ad0-a347-d89681f0356b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5418119f4564ac48a37f3821ced30f8', 'request_id': 'req-e83c0090-8328-4ad0-a347-d89681f0356b'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:52.037981 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-e83c0090-8328-4ad0-a347-d89681f0356b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5418119f4564ac48a37f3821ced30f8', 'request_id': 'req-e83c0090-8328-4ad0-a347-d89681f0356b'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:52.044531 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-e83c0090-8328-4ad0-a347-d89681f0356b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5418119f4564ac48a37f3821ced30f8', 'request_id': 'req-e83c0090-8328-4ad0-a347-d89681f0356b'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:52.057067 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-e83c0090-8328-4ad0-a347-d89681f0356b admin admin] The token cache is being invalidate because project f5418119f4564ac48a37f3821ced30f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80154) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:23:52.057354 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-e83c0090-8328-4ad0-a347-d89681f0356b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:52.057966 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-e83c0090-8328-4ad0-a347-d89681f0356b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:52.059558 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1308/2617] 10.222.0.64 () {58 vars in 1233 bytes} [Tue Aug 2 10:23:51 2022] DELETE /identity/v3/projects/f5418119f4564ac48a37f3821ced30f8 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:23:52.064138 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2483f22-e204-41ea-8074-78943c6b395d service neutron] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:52.074282 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2483f22-e204-41ea-8074-78943c6b395d service neutron] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:52.131287 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecde66df-683f-423d-8fd3-7d89062b7e39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:52.132051 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecde66df-683f-423d-8fd3-7d89062b7e39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:52.132479 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecde66df-683f-423d-8fd3-7d89062b7e39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:52.132946 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecde66df-683f-423d-8fd3-7d89062b7e39 admin admin] PATH_INFO: `/v3/users/c23b9651744c47139511d4d83114c044` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:52.134360 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecde66df-683f-423d-8fd3-7d89062b7e39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c23b9651744c47139511d4d83114c044)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:52.136309 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecde66df-683f-423d-8fd3-7d89062b7e39 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:52.182247 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-ecde66df-683f-423d-8fd3-7d89062b7e39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c23b9651744c47139511d4d83114c044'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:52.189234 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-ecde66df-683f-423d-8fd3-7d89062b7e39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c23b9651744c47139511d4d83114c044'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:52.196722 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-ecde66df-683f-423d-8fd3-7d89062b7e39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c23b9651744c47139511d4d83114c044'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:52.197357 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-ecde66df-683f-423d-8fd3-7d89062b7e39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c23b9651744c47139511d4d83114c044'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:52.197995 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-ecde66df-683f-423d-8fd3-7d89062b7e39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c23b9651744c47139511d4d83114c044'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:52.215404 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1310/2618] 10.222.0.64 () {58 vars in 1224 bytes} [Tue Aug 2 10:23:52 2022] DELETE /identity/v3/users/c23b9651744c47139511d4d83114c044 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:23:52.498445 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83c0090-8328-4ad0-a347-d89681f0356b admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:52.500214 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-e83c0090-8328-4ad0-a347-d89681f0356b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:52.517776 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83c0090-8328-4ad0-a347-d89681f0356b admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:52.580033 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f72a07-a87a-4d69-862f-aa059496e4e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:52.580703 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f72a07-a87a-4d69-862f-aa059496e4e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:52.581032 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f72a07-a87a-4d69-862f-aa059496e4e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:52.581331 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f72a07-a87a-4d69-862f-aa059496e4e4 admin admin] PATH_INFO: `/v3/projects/624aeca4fac34491a077be3a70494b9d` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:52.582633 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f72a07-a87a-4d69-862f-aa059496e4e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=624aeca4fac34491a077be3a70494b9d)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:52.584582 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f72a07-a87a-4d69-862f-aa059496e4e4 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:52.630781 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-12f72a07-a87a-4d69-862f-aa059496e4e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '624aeca4fac34491a077be3a70494b9d', 'request_id': 'req-12f72a07-a87a-4d69-862f-aa059496e4e4'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:52.636362 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-12f72a07-a87a-4d69-862f-aa059496e4e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '624aeca4fac34491a077be3a70494b9d', 'request_id': 'req-12f72a07-a87a-4d69-862f-aa059496e4e4'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:52.643217 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-12f72a07-a87a-4d69-862f-aa059496e4e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '624aeca4fac34491a077be3a70494b9d', 'request_id': 'req-12f72a07-a87a-4d69-862f-aa059496e4e4'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:52.656307 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-12f72a07-a87a-4d69-862f-aa059496e4e4 admin admin] The token cache is being invalidate because project 624aeca4fac34491a077be3a70494b9d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80154) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:23:52.656598 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-12f72a07-a87a-4d69-862f-aa059496e4e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:52.657240 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-12f72a07-a87a-4d69-862f-aa059496e4e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:23:52.658742 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1309/2619] 10.222.0.64 () {58 vars in 1233 bytes} [Tue Aug 2 10:23:52 2022] DELETE /identity/v3/projects/624aeca4fac34491a077be3a70494b9d => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:23:52.663802 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b56073-99ac-479e-ac3a-e832b2561773 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:52.664302 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b56073-99ac-479e-ac3a-e832b2561773 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:52.664423 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b56073-99ac-479e-ac3a-e832b2561773 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:52.684769 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-28b56073-99ac-479e-ac3a-e832b2561773 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:23:52.686020 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-28b56073-99ac-479e-ac3a-e832b2561773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:52.706521 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:23:52.749961 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1311/2620] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:23:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:23:52.757763 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d12f8e-67af-4cda-9904-bf7f059b60ef None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:52.758050 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d12f8e-67af-4cda-9904-bf7f059b60ef None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:52.758326 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d12f8e-67af-4cda-9904-bf7f059b60ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:52.782610 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-b0d12f8e-67af-4cda-9904-bf7f059b60ef None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:23:52.783831 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-b0d12f8e-67af-4cda-9904-bf7f059b60ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:52.793864 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1310/2621] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:23:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:23:52.797882 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b56073-99ac-479e-ac3a-e832b2561773 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:52.808847 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b56073-99ac-479e-ac3a-e832b2561773 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:52.814144 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09bf23a-51df-4a28-bda8-f133f9d4a14e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:52.814773 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09bf23a-51df-4a28-bda8-f133f9d4a14e admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:52.815061 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09bf23a-51df-4a28-bda8-f133f9d4a14e admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:52.815337 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09bf23a-51df-4a28-bda8-f133f9d4a14e admin admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:52.816053 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09bf23a-51df-4a28-bda8-f133f9d4a14e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:23:52.816394 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09bf23a-51df-4a28-bda8-f133f9d4a14e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:52.818011 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09bf23a-51df-4a28-bda8-f133f9d4a14e admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:52.833036 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1312/2622] 10.222.0.64 () {58 vars in 1153 bytes} [Tue Aug 2 10:23:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:23:52.837215 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d12f8e-67af-4cda-9904-bf7f059b60ef None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:52.852692 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d12f8e-67af-4cda-9904-bf7f059b60ef None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:52.856841 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2294228b-bff7-4840-a37a-5d6dc8ef3ff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:52.857468 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2294228b-bff7-4840-a37a-5d6dc8ef3ff6 admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:52.857748 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2294228b-bff7-4840-a37a-5d6dc8ef3ff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:52.858053 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2294228b-bff7-4840-a37a-5d6dc8ef3ff6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:52.858738 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2294228b-bff7-4840-a37a-5d6dc8ef3ff6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:52.860694 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2294228b-bff7-4840-a37a-5d6dc8ef3ff6 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:52.891943 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:23:52.901310 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1311/2623] 10.222.0.64 () {60 vars in 1153 bytes} [Tue Aug 2 10:23:52 2022] POST /identity/v3/projects => generated 383 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:23:52.905663 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09bf23a-51df-4a28-bda8-f133f9d4a14e admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:52.917462 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09bf23a-51df-4a28-bda8-f133f9d4a14e admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:52.986295 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7b7875-4f5d-49b7-b062-9df15c536a99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:52.986938 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7b7875-4f5d-49b7-b062-9df15c536a99 admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:52.987222 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7b7875-4f5d-49b7-b062-9df15c536a99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:52.987526 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7b7875-4f5d-49b7-b062-9df15c536a99 admin admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:52.988204 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7b7875-4f5d-49b7-b062-9df15c536a99 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:52.991086 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7b7875-4f5d-49b7-b062-9df15c536a99 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:53.020163 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1313/2624] 10.222.0.64 () {60 vars in 1144 bytes} [Tue Aug 2 10:23:52 2022] POST /identity/v3/users => generated 292 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:23:53.022061 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2294228b-bff7-4840-a37a-5d6dc8ef3ff6 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:53.034629 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2294228b-bff7-4840-a37a-5d6dc8ef3ff6 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:53.039530 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5938bb43-37ed-4fa9-80d8-0acfe2ffde12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:53.040134 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5938bb43-37ed-4fa9-80d8-0acfe2ffde12 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:53.040451 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5938bb43-37ed-4fa9-80d8-0acfe2ffde12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:53.040722 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5938bb43-37ed-4fa9-80d8-0acfe2ffde12 admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:53.041426 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5938bb43-37ed-4fa9-80d8-0acfe2ffde12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:53.043013 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5938bb43-37ed-4fa9-80d8-0acfe2ffde12 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:53.055816 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1312/2625] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:23:53 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:23:53.060389 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7b7875-4f5d-49b7-b062-9df15c536a99 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:53.072627 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7b7875-4f5d-49b7-b062-9df15c536a99 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:53.077956 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba52134-2729-4f94-930b-9c4219fadd88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:53.078590 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba52134-2729-4f94-930b-9c4219fadd88 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:53.078874 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba52134-2729-4f94-930b-9c4219fadd88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:53.079145 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba52134-2729-4f94-930b-9c4219fadd88 admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:53.079844 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba52134-2729-4f94-930b-9c4219fadd88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:53.081488 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba52134-2729-4f94-930b-9c4219fadd88 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:53.093080 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1314/2626] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:23:53 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:23:53.097486 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5938bb43-37ed-4fa9-80d8-0acfe2ffde12 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:53.107751 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5938bb43-37ed-4fa9-80d8-0acfe2ffde12 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:53.112500 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81a0c1e-0575-44bd-815d-e839127e4fd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:53.114120 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81a0c1e-0575-44bd-815d-e839127e4fd9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:53.114120 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81a0c1e-0575-44bd-815d-e839127e4fd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:53.114120 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81a0c1e-0575-44bd-815d-e839127e4fd9 admin admin] PATH_INFO: `/v3/projects/307655b70b684fb8bbee7a37c42cbb6a/users/77dc166f08a44bb890188d8b71ceba12/roles/66c1074780954f8e87ed11a5c4e8c36a` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:53.134597 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81a0c1e-0575-44bd-815d-e839127e4fd9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=307655b70b684fb8bbee7a37c42cbb6a, user_id=77dc166f08a44bb890188d8b71ceba12, role_id=66c1074780954f8e87ed11a5c4e8c36a)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:53.137407 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81a0c1e-0575-44bd-815d-e839127e4fd9 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:53.155453 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1313/2627] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 10:23:53 2022] PUT /identity/v3/projects/307655b70b684fb8bbee7a37c42cbb6a/users/77dc166f08a44bb890188d8b71ceba12/roles/66c1074780954f8e87ed11a5c4e8c36a => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:23:53.166889 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2c3d66-fd9b-4116-8d6e-3e53fb3e7dfa None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:53.167224 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2c3d66-fd9b-4116-8d6e-3e53fb3e7dfa None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:53.167939 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2c3d66-fd9b-4116-8d6e-3e53fb3e7dfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:53.199122 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-8e2c3d66-fd9b-4116-8d6e-3e53fb3e7dfa None None] MFA Rules not processed for user `77dc166f08a44bb890188d8b71ceba12`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:23:53.200393 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-8e2c3d66-fd9b-4116-8d6e-3e53fb3e7dfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:53.267512 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1315/2628] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:23:53 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:23:53.279361 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81a0c1e-0575-44bd-815d-e839127e4fd9 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:23:53.280282 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-a81a0c1e-0575-44bd-815d-e839127e4fd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:23:53.341087 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81a0c1e-0575-44bd-815d-e839127e4fd9 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:23:53.344795 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0906c68-dce9-4406-a40d-b67f9e0541af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d7de62b9ee4039bdc79627ac7fb499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:23:53.345413 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0906c68-dce9-4406-a40d-b67f9e0541af service cinder] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:23:53.345694 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0906c68-dce9-4406-a40d-b67f9e0541af service cinder] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:23:53.345968 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0906c68-dce9-4406-a40d-b67f9e0541af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:23:53.354845 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0906c68-dce9-4406-a40d-b67f9e0541af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:23:53.356314 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0906c68-dce9-4406-a40d-b67f9e0541af service cinder] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:23:53.367790 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1314/2629] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:23:53 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:24:03.263717 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=88,keystone:INSERT=3 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:24:03.361715 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=42,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:25:46.328950 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2c3d66-fd9b-4116-8d6e-3e53fb3e7dfa None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:25:46.338526 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2c3d66-fd9b-4116-8d6e-3e53fb3e7dfa None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:25:46.344094 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0799d8f7-0135-4419-a485-670715cae307 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d7de62b9ee4039bdc79627ac7fb499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:25:46.344730 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0799d8f7-0135-4419-a485-670715cae307 service cinder] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:25:46.345020 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0799d8f7-0135-4419-a485-670715cae307 service cinder] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:25:46.345294 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0799d8f7-0135-4419-a485-670715cae307 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:25:46.347044 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-0799d8f7-0135-4419-a485-670715cae307 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:25:46.419084 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0799d8f7-0135-4419-a485-670715cae307 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:25:46.420586 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0799d8f7-0135-4419-a485-670715cae307 service cinder] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:25:46.438621 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1316/2630] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:25:46 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:25:48.374228 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0906c68-dce9-4406-a40d-b67f9e0541af service cinder] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:25:48.376219 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-a0906c68-dce9-4406-a40d-b67f9e0541af service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:25:48.435880 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0906c68-dce9-4406-a40d-b67f9e0541af service cinder] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:25:48.439605 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a320f5-0884-47a9-a5c7-3806d66626ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:25:48.440272 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a320f5-0884-47a9-a5c7-3806d66626ab service neutron] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:25:48.440598 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a320f5-0884-47a9-a5c7-3806d66626ab service neutron] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:25:48.440875 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a320f5-0884-47a9-a5c7-3806d66626ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:25:48.442672 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-d3a320f5-0884-47a9-a5c7-3806d66626ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:25:48.533069 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a320f5-0884-47a9-a5c7-3806d66626ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:25:48.534588 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a320f5-0884-47a9-a5c7-3806d66626ab service neutron] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:25:48.547671 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1315/2631] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:25:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:25:48.986374 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0799d8f7-0135-4419-a485-670715cae307 service cinder] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:25:48.988391 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-0799d8f7-0135-4419-a485-670715cae307 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:25:49.005485 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0799d8f7-0135-4419-a485-670715cae307 service cinder] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:25:49.017084 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08d66b2-4f4a-44e3-8b85-15546a43961b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:25:49.017930 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08d66b2-4f4a-44e3-8b85-15546a43961b service neutron] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:25:49.018289 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08d66b2-4f4a-44e3-8b85-15546a43961b service neutron] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:25:49.018616 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08d66b2-4f4a-44e3-8b85-15546a43961b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:25:49.020894 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-b08d66b2-4f4a-44e3-8b85-15546a43961b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:25:49.035242 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08d66b2-4f4a-44e3-8b85-15546a43961b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:25:49.035242 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08d66b2-4f4a-44e3-8b85-15546a43961b service neutron] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:25:49.044811 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1317/2632] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:25:48 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:25:49.426895 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a320f5-0884-47a9-a5c7-3806d66626ab service neutron] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:25:49.430485 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-d3a320f5-0884-47a9-a5c7-3806d66626ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:25:49.470775 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a320f5-0884-47a9-a5c7-3806d66626ab service neutron] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:25:49.483915 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb0b31d-2efb-43e0-bac2-ed0a0fe0a01e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99655e1f9ca24472a86cd26b0dbd1fc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:25:49.484990 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb0b31d-2efb-43e0-bac2-ed0a0fe0a01e service nova] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:25:49.485527 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb0b31d-2efb-43e0-bac2-ed0a0fe0a01e service nova] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:25:49.486343 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb0b31d-2efb-43e0-bac2-ed0a0fe0a01e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:25:49.489963 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-1cb0b31d-2efb-43e0-bac2-ed0a0fe0a01e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:25:49.516209 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb0b31d-2efb-43e0-bac2-ed0a0fe0a01e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:25:49.518137 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb0b31d-2efb-43e0-bac2-ed0a0fe0a01e service nova] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:25:49.535161 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1316/2633] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:25:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:25:59.039723 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=39 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:25:59.528914 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=65 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:26:46.218297 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08d66b2-4f4a-44e3-8b85-15546a43961b service neutron] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:26:46.227916 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08d66b2-4f4a-44e3-8b85-15546a43961b service neutron] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:26:46.233452 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b1e92b-94d9-463d-b29e-e924b1cd6430 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:26:46.233968 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b1e92b-94d9-463d-b29e-e924b1cd6430 service neutron] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:26:46.234251 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b1e92b-94d9-463d-b29e-e924b1cd6430 service neutron] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:26:46.234532 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b1e92b-94d9-463d-b29e-e924b1cd6430 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:26:46.236912 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-08b1e92b-94d9-463d-b29e-e924b1cd6430 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:26:46.255877 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b1e92b-94d9-463d-b29e-e924b1cd6430 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:26:46.257805 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b1e92b-94d9-463d-b29e-e924b1cd6430 service neutron] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:26:46.273904 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1318/2634] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:26:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:26:56.266853 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=6 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:27:47.781028 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb0b31d-2efb-43e0-bac2-ed0a0fe0a01e service nova] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:27:47.791902 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb0b31d-2efb-43e0-bac2-ed0a0fe0a01e service nova] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:27:47.795793 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b3e0b9-f02b-4883-a588-cda5342f734c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d7de62b9ee4039bdc79627ac7fb499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:27:47.796466 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b3e0b9-f02b-4883-a588-cda5342f734c service cinder] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:27:47.796751 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b3e0b9-f02b-4883-a588-cda5342f734c service cinder] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:27:47.797024 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b3e0b9-f02b-4883-a588-cda5342f734c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:27:47.799182 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-52b3e0b9-f02b-4883-a588-cda5342f734c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:27:47.879607 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b3e0b9-f02b-4883-a588-cda5342f734c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:27:47.882935 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b3e0b9-f02b-4883-a588-cda5342f734c service cinder] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:27:47.891977 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1317/2635] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:27:47 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:27:57.077167 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b1e92b-94d9-463d-b29e-e924b1cd6430 service neutron] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:27:57.078548 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-08b1e92b-94d9-463d-b29e-e924b1cd6430 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:27:57.197047 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b1e92b-94d9-463d-b29e-e924b1cd6430 service neutron] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:27:57.202115 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fbc418-7162-47c5-9a6f-9884a6beee98 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b770f073c86b4d88837a8b1f796a8b1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:27:57.202730 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fbc418-7162-47c5-9a6f-9884a6beee98 service placement] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:27:57.203013 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fbc418-7162-47c5-9a6f-9884a6beee98 service placement] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:27:57.203286 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fbc418-7162-47c5-9a6f-9884a6beee98 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:27:57.205095 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-11fbc418-7162-47c5-9a6f-9884a6beee98 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:27:57.216575 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fbc418-7162-47c5-9a6f-9884a6beee98 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:27:57.218156 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fbc418-7162-47c5-9a6f-9884a6beee98 service placement] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:27:57.228412 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1319/2636] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:27:57 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:27:57.887112 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=33 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:28:07.223810 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=54 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:28:45.935085 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b3e0b9-f02b-4883-a588-cda5342f734c service cinder] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:28:45.986344 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b3e0b9-f02b-4883-a588-cda5342f734c service cinder] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:28:45.992646 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a29356-8b12-4cc6-93e4-8dc8f96a0acd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99655e1f9ca24472a86cd26b0dbd1fc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:28:45.993281 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a29356-8b12-4cc6-93e4-8dc8f96a0acd service nova] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:28:45.993568 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a29356-8b12-4cc6-93e4-8dc8f96a0acd service nova] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:28:45.993843 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a29356-8b12-4cc6-93e4-8dc8f96a0acd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:28:45.995598 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-13a29356-8b12-4cc6-93e4-8dc8f96a0acd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:28:46.052476 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a29356-8b12-4cc6-93e4-8dc8f96a0acd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:28:46.054629 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a29356-8b12-4cc6-93e4-8dc8f96a0acd service nova] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:28:46.063739 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1318/2637] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:28:45 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:28:56.059468 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=46 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:29:48.213895 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fbc418-7162-47c5-9a6f-9884a6beee98 service placement] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:29:48.233227 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fbc418-7162-47c5-9a6f-9884a6beee98 service placement] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:29:48.236886 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b195741-42aa-4b4f-a1a8-526ac6a4977a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d7de62b9ee4039bdc79627ac7fb499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:29:48.237499 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b195741-42aa-4b4f-a1a8-526ac6a4977a service cinder] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:29:48.237785 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b195741-42aa-4b4f-a1a8-526ac6a4977a service cinder] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:29:48.238078 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b195741-42aa-4b4f-a1a8-526ac6a4977a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:29:48.240534 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-2b195741-42aa-4b4f-a1a8-526ac6a4977a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:29:48.299650 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b195741-42aa-4b4f-a1a8-526ac6a4977a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:29:48.301710 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b195741-42aa-4b4f-a1a8-526ac6a4977a service cinder] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:29:48.310843 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1320/2638] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:29:48 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:29:48.375098 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a29356-8b12-4cc6-93e4-8dc8f96a0acd service nova] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:29:48.384449 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a29356-8b12-4cc6-93e4-8dc8f96a0acd service nova] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:29:48.387953 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e182663a-676c-4c11-8b8c-dbb0d47ca937 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d7de62b9ee4039bdc79627ac7fb499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:29:48.388665 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e182663a-676c-4c11-8b8c-dbb0d47ca937 service cinder] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:29:48.388946 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e182663a-676c-4c11-8b8c-dbb0d47ca937 service cinder] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:29:48.389218 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e182663a-676c-4c11-8b8c-dbb0d47ca937 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:29:48.398527 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e182663a-676c-4c11-8b8c-dbb0d47ca937 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:29:48.399976 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e182663a-676c-4c11-8b8c-dbb0d47ca937 service cinder] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:29:48.409322 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1319/2639] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:29:48 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:29:48.661011 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b195741-42aa-4b4f-a1a8-526ac6a4977a service cinder] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:29:48.671730 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b195741-42aa-4b4f-a1a8-526ac6a4977a service cinder] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:29:48.675279 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ad3c3b-4528-4770-86d3-abf13852e75e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:29:48.675938 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ad3c3b-4528-4770-86d3-abf13852e75e service neutron] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:29:48.676224 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ad3c3b-4528-4770-86d3-abf13852e75e service neutron] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:29:48.676531 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ad3c3b-4528-4770-86d3-abf13852e75e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:29:48.678304 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-f1ad3c3b-4528-4770-86d3-abf13852e75e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:29:48.754716 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ad3c3b-4528-4770-86d3-abf13852e75e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:29:48.756213 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ad3c3b-4528-4770-86d3-abf13852e75e service neutron] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:29:48.766671 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1321/2640] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:29:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:29:52.201160 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e182663a-676c-4c11-8b8c-dbb0d47ca937 service cinder] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:29:52.213681 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e182663a-676c-4c11-8b8c-dbb0d47ca937 service cinder] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:29:52.218754 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5a1d7e-e84c-42d2-9077-80f7df2bc94e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b770f073c86b4d88837a8b1f796a8b1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:29:52.219414 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5a1d7e-e84c-42d2-9077-80f7df2bc94e service placement] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:29:52.219702 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5a1d7e-e84c-42d2-9077-80f7df2bc94e service placement] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:29:52.219980 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5a1d7e-e84c-42d2-9077-80f7df2bc94e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:29:52.221846 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-ba5a1d7e-e84c-42d2-9077-80f7df2bc94e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:29:52.234657 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5a1d7e-e84c-42d2-9077-80f7df2bc94e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:29:52.236358 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5a1d7e-e84c-42d2-9077-80f7df2bc94e service placement] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:29:52.248519 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1320/2641] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:29:52 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:29:52.255792 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ad3c3b-4528-4770-86d3-abf13852e75e service neutron] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:29:52.257554 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-f1ad3c3b-4528-4770-86d3-abf13852e75e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:29:52.273478 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ad3c3b-4528-4770-86d3-abf13852e75e service neutron] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:29:52.278776 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac66544e-5e28-48b7-8a19-729597650472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:29:52.279434 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac66544e-5e28-48b7-8a19-729597650472 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:29:52.279723 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac66544e-5e28-48b7-8a19-729597650472 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:29:52.280003 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac66544e-5e28-48b7-8a19-729597650472 admin admin] PATH_INFO: `/v3/users/c94f607a834f43e7a0073fe27d265901` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:29:52.281439 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac66544e-5e28-48b7-8a19-729597650472 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c94f607a834f43e7a0073fe27d265901)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:29:52.283374 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac66544e-5e28-48b7-8a19-729597650472 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:29:52.335050 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-ac66544e-5e28-48b7-8a19-729597650472 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c94f607a834f43e7a0073fe27d265901'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:29:52.342451 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-ac66544e-5e28-48b7-8a19-729597650472 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c94f607a834f43e7a0073fe27d265901'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:29:52.350579 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-ac66544e-5e28-48b7-8a19-729597650472 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c94f607a834f43e7a0073fe27d265901'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:29:52.351256 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-ac66544e-5e28-48b7-8a19-729597650472 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c94f607a834f43e7a0073fe27d265901'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:29:52.351872 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-ac66544e-5e28-48b7-8a19-729597650472 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c94f607a834f43e7a0073fe27d265901'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:29:52.362787 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:29:52.370940 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1322/2642] 10.222.0.64 () {58 vars in 1224 bytes} [Tue Aug 2 10:29:52 2022] DELETE /identity/v3/users/c94f607a834f43e7a0073fe27d265901 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:29:52.553212 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5a1d7e-e84c-42d2-9077-80f7df2bc94e service placement] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:29:52.564358 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5a1d7e-e84c-42d2-9077-80f7df2bc94e service placement] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:29:52.623846 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5ddb37-1c7a-4d3e-86fa-e81422e20177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:29:52.624522 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5ddb37-1c7a-4d3e-86fa-e81422e20177 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:29:52.624804 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5ddb37-1c7a-4d3e-86fa-e81422e20177 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:29:52.625075 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5ddb37-1c7a-4d3e-86fa-e81422e20177 admin admin] PATH_INFO: `/v3/projects/d0b63cdea5bf43ea8478b59c7154ee70` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:29:52.626249 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5ddb37-1c7a-4d3e-86fa-e81422e20177 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0b63cdea5bf43ea8478b59c7154ee70)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:29:52.628153 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5ddb37-1c7a-4d3e-86fa-e81422e20177 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:29:52.677065 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-3a5ddb37-1c7a-4d3e-86fa-e81422e20177 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0b63cdea5bf43ea8478b59c7154ee70', 'request_id': 'req-3a5ddb37-1c7a-4d3e-86fa-e81422e20177'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:29:52.682695 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-3a5ddb37-1c7a-4d3e-86fa-e81422e20177 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0b63cdea5bf43ea8478b59c7154ee70', 'request_id': 'req-3a5ddb37-1c7a-4d3e-86fa-e81422e20177'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:29:52.689291 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-3a5ddb37-1c7a-4d3e-86fa-e81422e20177 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0b63cdea5bf43ea8478b59c7154ee70', 'request_id': 'req-3a5ddb37-1c7a-4d3e-86fa-e81422e20177'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:29:52.702791 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-3a5ddb37-1c7a-4d3e-86fa-e81422e20177 admin admin] The token cache is being invalidate because project d0b63cdea5bf43ea8478b59c7154ee70 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80154) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:29:52.703013 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-3a5ddb37-1c7a-4d3e-86fa-e81422e20177 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:29:52.703635 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-3a5ddb37-1c7a-4d3e-86fa-e81422e20177 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:29:52.705195 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1321/2643] 10.222.0.64 () {58 vars in 1233 bytes} [Tue Aug 2 10:29:52 2022] DELETE /identity/v3/projects/d0b63cdea5bf43ea8478b59c7154ee70 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:29:52.712269 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b45040-0613-4223-87b4-2e7b6d4e14f5 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:29:52.712602 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b45040-0613-4223-87b4-2e7b6d4e14f5 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:29:52.712912 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b45040-0613-4223-87b4-2e7b6d4e14f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:29:52.732375 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-88b45040-0613-4223-87b4-2e7b6d4e14f5 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:29:52.733669 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-88b45040-0613-4223-87b4-2e7b6d4e14f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:29:52.791997 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1323/2644] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:29:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:29:52.801515 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aaad39-2c72-4954-be62-8af8815dede6 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:29:52.801910 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aaad39-2c72-4954-be62-8af8815dede6 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:29:52.802255 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aaad39-2c72-4954-be62-8af8815dede6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:29:52.822415 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-f4aaad39-2c72-4954-be62-8af8815dede6 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:29:52.823625 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-f4aaad39-2c72-4954-be62-8af8815dede6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:29:52.831480 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1322/2645] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:29:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:29:52.836445 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b45040-0613-4223-87b4-2e7b6d4e14f5 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:29:52.846222 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b45040-0613-4223-87b4-2e7b6d4e14f5 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:29:52.850972 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a925aef-e31d-4025-972a-93eb7b1ae3a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:29:52.851586 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a925aef-e31d-4025-972a-93eb7b1ae3a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:29:52.851868 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a925aef-e31d-4025-972a-93eb7b1ae3a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:29:52.852139 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a925aef-e31d-4025-972a-93eb7b1ae3a6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:29:52.852902 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a925aef-e31d-4025-972a-93eb7b1ae3a6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:29:52.853207 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a925aef-e31d-4025-972a-93eb7b1ae3a6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:29:52.854796 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a925aef-e31d-4025-972a-93eb7b1ae3a6 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:29:52.867080 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1324/2646] 10.222.0.64 () {58 vars in 1153 bytes} [Tue Aug 2 10:29:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:29:52.871819 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4aaad39-2c72-4954-be62-8af8815dede6 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:29:52.882338 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4aaad39-2c72-4954-be62-8af8815dede6 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:29:52.887166 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67beeb53-c34b-4c42-a483-44c5784f1520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:29:52.887796 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67beeb53-c34b-4c42-a483-44c5784f1520 admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:29:52.888076 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67beeb53-c34b-4c42-a483-44c5784f1520 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:29:52.888379 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67beeb53-c34b-4c42-a483-44c5784f1520 admin admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:29:52.889051 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67beeb53-c34b-4c42-a483-44c5784f1520 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:29:52.891019 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67beeb53-c34b-4c42-a483-44c5784f1520 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:29:52.918319 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1323/2647] 10.222.0.64 () {60 vars in 1153 bytes} [Tue Aug 2 10:29:52 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:29:52.922619 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a925aef-e31d-4025-972a-93eb7b1ae3a6 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:29:52.933083 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a925aef-e31d-4025-972a-93eb7b1ae3a6 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:29:52.989134 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4244af5-9f02-439f-a7ee-276142913f6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:29:52.989656 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4244af5-9f02-439f-a7ee-276142913f6c admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:29:52.989938 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4244af5-9f02-439f-a7ee-276142913f6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:29:52.990212 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4244af5-9f02-439f-a7ee-276142913f6c admin admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:29:52.990880 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4244af5-9f02-439f-a7ee-276142913f6c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:29:52.992823 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4244af5-9f02-439f-a7ee-276142913f6c admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:29:53.014913 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1325/2648] 10.222.0.64 () {60 vars in 1144 bytes} [Tue Aug 2 10:29:52 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:29:53.019162 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67beeb53-c34b-4c42-a483-44c5784f1520 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:29:53.029769 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67beeb53-c34b-4c42-a483-44c5784f1520 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:29:53.034497 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bb6420-b763-4dfd-aea3-f758affaa78c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:29:53.035109 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bb6420-b763-4dfd-aea3-f758affaa78c admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:29:53.035395 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bb6420-b763-4dfd-aea3-f758affaa78c admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:29:53.035669 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bb6420-b763-4dfd-aea3-f758affaa78c admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:29:53.036403 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bb6420-b763-4dfd-aea3-f758affaa78c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:29:53.038386 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bb6420-b763-4dfd-aea3-f758affaa78c admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:29:53.048475 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1324/2649] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:29:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:29:53.053178 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4244af5-9f02-439f-a7ee-276142913f6c admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:29:53.063295 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4244af5-9f02-439f-a7ee-276142913f6c admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:29:53.068798 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b580dc-110a-460b-a487-c0f568a49c11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:29:53.069401 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b580dc-110a-460b-a487-c0f568a49c11 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:29:53.069682 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b580dc-110a-460b-a487-c0f568a49c11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:29:53.069955 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b580dc-110a-460b-a487-c0f568a49c11 admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:29:53.070645 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b580dc-110a-460b-a487-c0f568a49c11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:29:53.072287 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b580dc-110a-460b-a487-c0f568a49c11 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:29:53.083001 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1326/2650] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:29:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:29:53.087216 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bb6420-b763-4dfd-aea3-f758affaa78c admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:29:53.097300 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bb6420-b763-4dfd-aea3-f758affaa78c admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:29:53.101927 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84357621-8993-4afe-b9fd-b26c950f911a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:29:53.102584 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84357621-8993-4afe-b9fd-b26c950f911a admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:29:53.102869 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84357621-8993-4afe-b9fd-b26c950f911a admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:29:53.103148 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84357621-8993-4afe-b9fd-b26c950f911a admin admin] PATH_INFO: `/v3/projects/47918b1c54ec4aa89c545d353d215607/users/8648997bf91c44a4bef3893d332a3254/roles/66c1074780954f8e87ed11a5c4e8c36a` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:29:53.118037 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84357621-8993-4afe-b9fd-b26c950f911a admin admin] RBAC: Authorizing `identity:create_grant(project_id=47918b1c54ec4aa89c545d353d215607, user_id=8648997bf91c44a4bef3893d332a3254, role_id=66c1074780954f8e87ed11a5c4e8c36a)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:29:53.120713 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84357621-8993-4afe-b9fd-b26c950f911a admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:29:53.130026 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:29:53.135724 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1325/2651] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 10:29:53 2022] PUT /identity/v3/projects/47918b1c54ec4aa89c545d353d215607/users/8648997bf91c44a4bef3893d332a3254/roles/66c1074780954f8e87ed11a5c4e8c36a => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:29:53.146186 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37dc9bd-5dc5-46b1-92be-ae0c925cd30f None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:29:53.146475 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37dc9bd-5dc5-46b1-92be-ae0c925cd30f None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:29:53.146753 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37dc9bd-5dc5-46b1-92be-ae0c925cd30f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:29:53.175954 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-d37dc9bd-5dc5-46b1-92be-ae0c925cd30f None None] MFA Rules not processed for user `8648997bf91c44a4bef3893d332a3254`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:29:53.177378 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-d37dc9bd-5dc5-46b1-92be-ae0c925cd30f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:29:53.227816 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:29:53.250348 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1327/2652] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:29:53 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:29:53.261569 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84357621-8993-4afe-b9fd-b26c950f911a admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:29:53.263464 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-84357621-8993-4afe-b9fd-b26c950f911a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:29:53.336612 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84357621-8993-4afe-b9fd-b26c950f911a admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:29:53.342069 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfc3886-3e8f-40c4-a55a-41ff164b838b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99655e1f9ca24472a86cd26b0dbd1fc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:29:53.342683 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfc3886-3e8f-40c4-a55a-41ff164b838b service nova] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:29:53.342967 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfc3886-3e8f-40c4-a55a-41ff164b838b service nova] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:29:53.343241 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfc3886-3e8f-40c4-a55a-41ff164b838b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:29:53.353071 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfc3886-3e8f-40c4-a55a-41ff164b838b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:29:53.354772 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfc3886-3e8f-40c4-a55a-41ff164b838b service nova] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:29:53.364362 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1326/2653] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:29:53 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:30:03.247102 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=4 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:30:03.359651 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=34 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:30:46.740009 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37dc9bd-5dc5-46b1-92be-ae0c925cd30f None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:30:46.741514 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-d37dc9bd-5dc5-46b1-92be-ae0c925cd30f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:30:46.758276 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37dc9bd-5dc5-46b1-92be-ae0c925cd30f None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:30:46.763949 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04be2dc3-bf03-41f2-b6a2-db45360fb890 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99655e1f9ca24472a86cd26b0dbd1fc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:30:46.764604 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04be2dc3-bf03-41f2-b6a2-db45360fb890 service nova] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:30:46.764888 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04be2dc3-bf03-41f2-b6a2-db45360fb890 service nova] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:30:46.765163 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04be2dc3-bf03-41f2-b6a2-db45360fb890 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:30:46.767091 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-04be2dc3-bf03-41f2-b6a2-db45360fb890 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:30:46.846924 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04be2dc3-bf03-41f2-b6a2-db45360fb890 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:30:46.848789 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04be2dc3-bf03-41f2-b6a2-db45360fb890 service nova] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:30:46.869170 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1328/2654] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:30:46 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:30:48.105303 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfc3886-3e8f-40c4-a55a-41ff164b838b service nova] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:30:48.106826 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-3cfc3886-3e8f-40c4-a55a-41ff164b838b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:30:48.173903 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfc3886-3e8f-40c4-a55a-41ff164b838b service nova] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:30:48.177471 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6c9a75-0a2f-4a5d-9aa2-c27fbfed0b85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:30:48.178087 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6c9a75-0a2f-4a5d-9aa2-c27fbfed0b85 service neutron] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:30:48.178386 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6c9a75-0a2f-4a5d-9aa2-c27fbfed0b85 service neutron] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:30:48.178637 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6c9a75-0a2f-4a5d-9aa2-c27fbfed0b85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:30:48.180431 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-1e6c9a75-0a2f-4a5d-9aa2-c27fbfed0b85 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:30:48.194849 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6c9a75-0a2f-4a5d-9aa2-c27fbfed0b85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:30:48.196303 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6c9a75-0a2f-4a5d-9aa2-c27fbfed0b85 service neutron] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:30:48.208138 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1327/2655] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:30:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:30:53.318907 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04be2dc3-bf03-41f2-b6a2-db45360fb890 service nova] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:30:53.382164 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04be2dc3-bf03-41f2-b6a2-db45360fb890 service nova] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:30:53.386739 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab4c1b6-208b-4ae3-b9d0-e52d576c017a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:30:53.387381 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab4c1b6-208b-4ae3-b9d0-e52d576c017a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:30:53.387669 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab4c1b6-208b-4ae3-b9d0-e52d576c017a admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:30:53.387957 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab4c1b6-208b-4ae3-b9d0-e52d576c017a admin admin] PATH_INFO: `/v3/users/8648997bf91c44a4bef3893d332a3254` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:30:53.389158 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab4c1b6-208b-4ae3-b9d0-e52d576c017a admin admin] RBAC: Authorizing `identity:delete_user(user_id=8648997bf91c44a4bef3893d332a3254)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:30:53.391091 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab4c1b6-208b-4ae3-b9d0-e52d576c017a admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:30:53.438439 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-6ab4c1b6-208b-4ae3-b9d0-e52d576c017a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8648997bf91c44a4bef3893d332a3254'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:30:53.446029 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-6ab4c1b6-208b-4ae3-b9d0-e52d576c017a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8648997bf91c44a4bef3893d332a3254'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:30:53.453657 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-6ab4c1b6-208b-4ae3-b9d0-e52d576c017a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8648997bf91c44a4bef3893d332a3254'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:30:53.454324 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-6ab4c1b6-208b-4ae3-b9d0-e52d576c017a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8648997bf91c44a4bef3893d332a3254'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:30:53.455499 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-6ab4c1b6-208b-4ae3-b9d0-e52d576c017a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8648997bf91c44a4bef3893d332a3254'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:30:53.468869 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:30:53.473617 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1329/2656] 10.222.0.64 () {58 vars in 1224 bytes} [Tue Aug 2 10:30:53 2022] DELETE /identity/v3/users/8648997bf91c44a4bef3893d332a3254 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:30:53.478521 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f67802-e141-4c6b-8658-597bff7e9920 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:30:53.479078 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f67802-e141-4c6b-8658-597bff7e9920 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:30:53.479078 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f67802-e141-4c6b-8658-597bff7e9920 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:30:53.497660 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-f7f67802-e141-4c6b-8658-597bff7e9920 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:30:53.498960 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-f7f67802-e141-4c6b-8658-597bff7e9920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:30:53.556644 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1328/2657] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:30:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:30:53.567281 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab4c1b6-208b-4ae3-b9d0-e52d576c017a admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:30:53.568709 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-6ab4c1b6-208b-4ae3-b9d0-e52d576c017a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:30:53.626063 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab4c1b6-208b-4ae3-b9d0-e52d576c017a admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:30:53.630316 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c549d6-44ea-4819-b42b-e5020f624ad0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:30:53.630936 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c549d6-44ea-4819-b42b-e5020f624ad0 service neutron] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:30:53.631220 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c549d6-44ea-4819-b42b-e5020f624ad0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:30:53.631496 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c549d6-44ea-4819-b42b-e5020f624ad0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:30:53.640729 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c549d6-44ea-4819-b42b-e5020f624ad0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:30:53.642194 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c549d6-44ea-4819-b42b-e5020f624ad0 service neutron] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:30:53.651876 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1330/2658] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:30:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:30:53.752597 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f67802-e141-4c6b-8658-597bff7e9920 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:30:53.754125 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-f7f67802-e141-4c6b-8658-597bff7e9920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:30:53.767431 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f67802-e141-4c6b-8658-597bff7e9920 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:30:53.771129 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f55b32f-fe25-4ff9-a2be-ae5e8d9e8e64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:30:53.771756 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f55b32f-fe25-4ff9-a2be-ae5e8d9e8e64 service neutron] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:30:53.772042 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f55b32f-fe25-4ff9-a2be-ae5e8d9e8e64 service neutron] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:30:53.772354 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f55b32f-fe25-4ff9-a2be-ae5e8d9e8e64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:30:53.774899 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-9f55b32f-fe25-4ff9-a2be-ae5e8d9e8e64 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:30:53.784980 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f55b32f-fe25-4ff9-a2be-ae5e8d9e8e64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:30:53.786431 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f55b32f-fe25-4ff9-a2be-ae5e8d9e8e64 service neutron] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:30:53.795136 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1329/2659] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:30:53 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:30:53.967218 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c549d6-44ea-4819-b42b-e5020f624ad0 service neutron] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:30:53.969167 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-d5c549d6-44ea-4819-b42b-e5020f624ad0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:30:53.986721 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c549d6-44ea-4819-b42b-e5020f624ad0 service neutron] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:30:53.992362 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ee8cbf-8c7d-4774-a4ba-4755272a04ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:30:53.993132 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ee8cbf-8c7d-4774-a4ba-4755272a04ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:30:53.993428 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ee8cbf-8c7d-4774-a4ba-4755272a04ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:30:53.993736 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ee8cbf-8c7d-4774-a4ba-4755272a04ec admin admin] PATH_INFO: `/v3/projects/47918b1c54ec4aa89c545d353d215607` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:30:53.995857 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ee8cbf-8c7d-4774-a4ba-4755272a04ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=47918b1c54ec4aa89c545d353d215607)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:30:53.997937 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ee8cbf-8c7d-4774-a4ba-4755272a04ec admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:30:54.058163 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-24ee8cbf-8c7d-4774-a4ba-4755272a04ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47918b1c54ec4aa89c545d353d215607', 'request_id': 'req-24ee8cbf-8c7d-4774-a4ba-4755272a04ec'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:30:54.065600 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-24ee8cbf-8c7d-4774-a4ba-4755272a04ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47918b1c54ec4aa89c545d353d215607', 'request_id': 'req-24ee8cbf-8c7d-4774-a4ba-4755272a04ec'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:30:54.073085 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-24ee8cbf-8c7d-4774-a4ba-4755272a04ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47918b1c54ec4aa89c545d353d215607', 'request_id': 'req-24ee8cbf-8c7d-4774-a4ba-4755272a04ec'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:30:54.088029 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-24ee8cbf-8c7d-4774-a4ba-4755272a04ec admin admin] The token cache is being invalidate because project 47918b1c54ec4aa89c545d353d215607 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80153) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:30:54.088357 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-24ee8cbf-8c7d-4774-a4ba-4755272a04ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:30:54.089053 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-24ee8cbf-8c7d-4774-a4ba-4755272a04ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:30:54.091080 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1331/2660] 10.222.0.64 () {58 vars in 1233 bytes} [Tue Aug 2 10:30:53 2022] DELETE /identity/v3/projects/47918b1c54ec4aa89c545d353d215607 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:30:54.096868 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd397ff-624c-454c-9b25-f38610d8e856 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:30:54.097436 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd397ff-624c-454c-9b25-f38610d8e856 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:30:54.097784 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd397ff-624c-454c-9b25-f38610d8e856 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:30:54.119285 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-2dd397ff-624c-454c-9b25-f38610d8e856 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:30:54.120670 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-2dd397ff-624c-454c-9b25-f38610d8e856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:30:54.192375 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1330/2661] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:30:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:30:54.200024 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c48394-e25d-439e-bfaf-ff8b3be11597 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:30:54.200509 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c48394-e25d-439e-bfaf-ff8b3be11597 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:30:54.200797 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c48394-e25d-439e-bfaf-ff8b3be11597 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:30:54.225401 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-63c48394-e25d-439e-bfaf-ff8b3be11597 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:30:54.226637 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-63c48394-e25d-439e-bfaf-ff8b3be11597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:30:54.235245 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1332/2662] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:30:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:30:54.239925 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd397ff-624c-454c-9b25-f38610d8e856 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:30:54.250888 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd397ff-624c-454c-9b25-f38610d8e856 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:30:54.256124 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efc2cec-e48e-40c2-a325-8bdb6c721e94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:30:54.256792 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efc2cec-e48e-40c2-a325-8bdb6c721e94 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:30:54.257075 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efc2cec-e48e-40c2-a325-8bdb6c721e94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:30:54.257505 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efc2cec-e48e-40c2-a325-8bdb6c721e94 admin admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:30:54.258320 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efc2cec-e48e-40c2-a325-8bdb6c721e94 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:30:54.258625 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efc2cec-e48e-40c2-a325-8bdb6c721e94 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:30:54.260281 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efc2cec-e48e-40c2-a325-8bdb6c721e94 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:30:54.276435 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1331/2663] 10.222.0.64 () {58 vars in 1153 bytes} [Tue Aug 2 10:30:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:30:54.278260 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c48394-e25d-439e-bfaf-ff8b3be11597 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:30:54.304318 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c48394-e25d-439e-bfaf-ff8b3be11597 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:30:54.304318 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddc4c3c-1d8f-440d-83f4-892cab5bf8b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:30:54.304318 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddc4c3c-1d8f-440d-83f4-892cab5bf8b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:30:54.304318 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddc4c3c-1d8f-440d-83f4-892cab5bf8b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:30:54.304948 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddc4c3c-1d8f-440d-83f4-892cab5bf8b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:30:54.304948 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddc4c3c-1d8f-440d-83f4-892cab5bf8b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:30:54.304948 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddc4c3c-1d8f-440d-83f4-892cab5bf8b4 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:30:54.331175 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1333/2664] 10.222.0.64 () {60 vars in 1153 bytes} [Tue Aug 2 10:30:54 2022] POST /identity/v3/projects => generated 373 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:30:54.335441 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efc2cec-e48e-40c2-a325-8bdb6c721e94 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:30:54.346871 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efc2cec-e48e-40c2-a325-8bdb6c721e94 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:30:54.352441 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:30:54.409083 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca90f47-cf91-4721-9b7a-8c6ba149b634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:30:54.410321 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca90f47-cf91-4721-9b7a-8c6ba149b634 admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:30:54.410321 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca90f47-cf91-4721-9b7a-8c6ba149b634 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:30:54.410321 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca90f47-cf91-4721-9b7a-8c6ba149b634 admin admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:30:54.410961 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca90f47-cf91-4721-9b7a-8c6ba149b634 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:30:54.412961 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca90f47-cf91-4721-9b7a-8c6ba149b634 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:30:54.435690 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1332/2665] 10.222.0.64 () {60 vars in 1144 bytes} [Tue Aug 2 10:30:54 2022] POST /identity/v3/users => generated 287 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:30:54.439946 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddc4c3c-1d8f-440d-83f4-892cab5bf8b4 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:30:54.451060 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddc4c3c-1d8f-440d-83f4-892cab5bf8b4 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:30:54.456580 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11c3b9a-29e1-43eb-be7a-0a918994be47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:30:54.457204 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11c3b9a-29e1-43eb-be7a-0a918994be47 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:30:54.457489 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11c3b9a-29e1-43eb-be7a-0a918994be47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:30:54.457764 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11c3b9a-29e1-43eb-be7a-0a918994be47 admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:30:54.458470 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11c3b9a-29e1-43eb-be7a-0a918994be47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:30:54.460078 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11c3b9a-29e1-43eb-be7a-0a918994be47 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:30:54.472411 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1334/2666] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:30:54 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:30:54.476409 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca90f47-cf91-4721-9b7a-8c6ba149b634 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:30:54.486792 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca90f47-cf91-4721-9b7a-8c6ba149b634 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:30:54.491829 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baf0adb-c85b-4474-bee9-72b5fdcc2d95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:30:54.492528 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baf0adb-c85b-4474-bee9-72b5fdcc2d95 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:30:54.492817 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baf0adb-c85b-4474-bee9-72b5fdcc2d95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:30:54.493094 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baf0adb-c85b-4474-bee9-72b5fdcc2d95 admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:30:54.493802 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5baf0adb-c85b-4474-bee9-72b5fdcc2d95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:30:54.495431 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5baf0adb-c85b-4474-bee9-72b5fdcc2d95 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:30:54.509994 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1333/2667] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:30:54 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:30:54.514251 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11c3b9a-29e1-43eb-be7a-0a918994be47 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:30:54.524864 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11c3b9a-29e1-43eb-be7a-0a918994be47 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:30:54.529383 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fa0a80-906a-42e2-b8a3-0b043f71decf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:30:54.530031 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fa0a80-906a-42e2-b8a3-0b043f71decf admin admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:30:54.530309 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fa0a80-906a-42e2-b8a3-0b043f71decf admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:30:54.530580 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fa0a80-906a-42e2-b8a3-0b043f71decf admin admin] PATH_INFO: `/v3/projects/98140239bedd487c9221e2252bbd0b6a/users/e620012d54c948e69764fda7e76238ac/roles/66c1074780954f8e87ed11a5c4e8c36a` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:30:54.545704 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fa0a80-906a-42e2-b8a3-0b043f71decf admin admin] RBAC: Authorizing `identity:create_grant(project_id=98140239bedd487c9221e2252bbd0b6a, user_id=e620012d54c948e69764fda7e76238ac, role_id=66c1074780954f8e87ed11a5c4e8c36a)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:30:54.548362 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fa0a80-906a-42e2-b8a3-0b043f71decf admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:30:54.561507 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1335/2668] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 10:30:54 2022] PUT /identity/v3/projects/98140239bedd487c9221e2252bbd0b6a/users/e620012d54c948e69764fda7e76238ac/roles/66c1074780954f8e87ed11a5c4e8c36a => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:30:54.573247 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54440bd4-e13d-421d-a2b6-f4da23861f48 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:30:54.573560 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54440bd4-e13d-421d-a2b6-f4da23861f48 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:30:54.573839 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54440bd4-e13d-421d-a2b6-f4da23861f48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:30:54.602180 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-54440bd4-e13d-421d-a2b6-f4da23861f48 None None] MFA Rules not processed for user `e620012d54c948e69764fda7e76238ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:30:54.603327 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-54440bd4-e13d-421d-a2b6-f4da23861f48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:30:54.670232 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1334/2669] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:30:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:30:54.681323 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fa0a80-906a-42e2-b8a3-0b043f71decf admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:30:54.683061 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-f0fa0a80-906a-42e2-b8a3-0b043f71decf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:30:54.709736 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:30:54.761257 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fa0a80-906a-42e2-b8a3-0b043f71decf admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:30:54.766728 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f19e8e-f2a7-49d9-b362-6b9a36bdde6d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99655e1f9ca24472a86cd26b0dbd1fc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:30:54.767468 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f19e8e-f2a7-49d9-b362-6b9a36bdde6d service nova] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:30:54.767751 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f19e8e-f2a7-49d9-b362-6b9a36bdde6d service nova] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:30:54.768023 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f19e8e-f2a7-49d9-b362-6b9a36bdde6d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:30:54.775915 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f19e8e-f2a7-49d9-b362-6b9a36bdde6d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:30:54.777561 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f19e8e-f2a7-49d9-b362-6b9a36bdde6d service nova] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:30:54.787379 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1336/2670] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:30:54 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:31:04.663613 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=62,keystone:INSERT=3 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:31:04.787820 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=24 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:31:48.691415 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54440bd4-e13d-421d-a2b6-f4da23861f48 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:31:48.692912 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-54440bd4-e13d-421d-a2b6-f4da23861f48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:31:48.752560 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54440bd4-e13d-421d-a2b6-f4da23861f48 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:31:48.756637 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e988b5-7464-41c2-89e8-c9f65521f15d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d7de62b9ee4039bdc79627ac7fb499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:31:48.757308 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e988b5-7464-41c2-89e8-c9f65521f15d service cinder] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:31:48.757628 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e988b5-7464-41c2-89e8-c9f65521f15d service cinder] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:31:48.757941 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e988b5-7464-41c2-89e8-c9f65521f15d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:31:48.759905 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-91e988b5-7464-41c2-89e8-c9f65521f15d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:31:48.775079 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e988b5-7464-41c2-89e8-c9f65521f15d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:31:48.776643 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e988b5-7464-41c2-89e8-c9f65521f15d service cinder] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:31:48.793327 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1335/2671] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:31:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:31:49.985986 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f19e8e-f2a7-49d9-b362-6b9a36bdde6d service nova] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:31:49.988191 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-18f19e8e-f2a7-49d9-b362-6b9a36bdde6d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:31:50.147462 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f19e8e-f2a7-49d9-b362-6b9a36bdde6d service nova] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:31:50.155373 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671c5594-7e63-4392-9ff6-70718742b4be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:31:50.156573 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671c5594-7e63-4392-9ff6-70718742b4be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:31:50.157551 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671c5594-7e63-4392-9ff6-70718742b4be admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:31:50.158041 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671c5594-7e63-4392-9ff6-70718742b4be admin admin] PATH_INFO: `/v3/users/77dc166f08a44bb890188d8b71ceba12` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:31:50.159379 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671c5594-7e63-4392-9ff6-70718742b4be admin admin] RBAC: Authorizing `identity:delete_user(user_id=77dc166f08a44bb890188d8b71ceba12)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:31:50.161384 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671c5594-7e63-4392-9ff6-70718742b4be admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:31:50.226203 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-671c5594-7e63-4392-9ff6-70718742b4be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77dc166f08a44bb890188d8b71ceba12'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:31:50.234677 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-671c5594-7e63-4392-9ff6-70718742b4be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77dc166f08a44bb890188d8b71ceba12'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:31:50.243887 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-671c5594-7e63-4392-9ff6-70718742b4be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77dc166f08a44bb890188d8b71ceba12'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:31:50.244751 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-671c5594-7e63-4392-9ff6-70718742b4be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77dc166f08a44bb890188d8b71ceba12'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:31:50.245483 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-671c5594-7e63-4392-9ff6-70718742b4be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77dc166f08a44bb890188d8b71ceba12'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:31:50.274755 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1337/2672] 10.222.0.64 () {58 vars in 1224 bytes} [Tue Aug 2 10:31:49 2022] DELETE /identity/v3/users/77dc166f08a44bb890188d8b71ceba12 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:31:50.283507 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f722ae-761b-4793-bf82-8d1a47857064 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:31:50.284285 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f722ae-761b-4793-bf82-8d1a47857064 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:31:50.284543 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f722ae-761b-4793-bf82-8d1a47857064 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:31:50.320536 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-f7f722ae-761b-4793-bf82-8d1a47857064 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:31:50.322734 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-f7f722ae-761b-4793-bf82-8d1a47857064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:31:50.436424 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1336/2673] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:31:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:31:50.449508 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671c5594-7e63-4392-9ff6-70718742b4be admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:31:50.454107 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-671c5594-7e63-4392-9ff6-70718742b4be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:31:50.611630 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671c5594-7e63-4392-9ff6-70718742b4be admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:31:50.642367 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cc5d85-bc4e-47a0-90a8-8792489214ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:31:50.643467 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cc5d85-bc4e-47a0-90a8-8792489214ce service neutron] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:31:50.646784 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cc5d85-bc4e-47a0-90a8-8792489214ce service neutron] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:31:50.646784 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cc5d85-bc4e-47a0-90a8-8792489214ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:31:50.675565 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cc5d85-bc4e-47a0-90a8-8792489214ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:31:50.678607 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cc5d85-bc4e-47a0-90a8-8792489214ce service neutron] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:31:50.700349 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1338/2674] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:31:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:31:50.789451 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f722ae-761b-4793-bf82-8d1a47857064 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:31:50.796489 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-f7f722ae-761b-4793-bf82-8d1a47857064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:31:50.975924 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f722ae-761b-4793-bf82-8d1a47857064 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:31:50.985518 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf0a28a-4ee3-458c-adf0-0a10427a7f11 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99655e1f9ca24472a86cd26b0dbd1fc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:31:50.986558 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf0a28a-4ee3-458c-adf0-0a10427a7f11 service nova] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:31:50.987045 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf0a28a-4ee3-458c-adf0-0a10427a7f11 service nova] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:31:50.987542 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf0a28a-4ee3-458c-adf0-0a10427a7f11 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:31:50.990645 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-ecf0a28a-4ee3-458c-adf0-0a10427a7f11 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:31:51.014992 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf0a28a-4ee3-458c-adf0-0a10427a7f11 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:31:51.018150 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf0a28a-4ee3-458c-adf0-0a10427a7f11 service nova] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:31:51.048437 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1337/2675] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:31:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:31:51.475586 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cc5d85-bc4e-47a0-90a8-8792489214ce service neutron] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:31:51.478354 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-a1cc5d85-bc4e-47a0-90a8-8792489214ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:31:51.507035 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cc5d85-bc4e-47a0-90a8-8792489214ce service neutron] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:31:51.514711 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48d3e05-9e1e-45ae-b425-ab40b7f83e6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:31:51.516789 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48d3e05-9e1e-45ae-b425-ab40b7f83e6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:31:51.517570 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48d3e05-9e1e-45ae-b425-ab40b7f83e6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:31:51.518268 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48d3e05-9e1e-45ae-b425-ab40b7f83e6d admin admin] PATH_INFO: `/v3/projects/307655b70b684fb8bbee7a37c42cbb6a` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:31:51.520671 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48d3e05-9e1e-45ae-b425-ab40b7f83e6d admin admin] RBAC: Authorizing `identity:delete_project(project_id=307655b70b684fb8bbee7a37c42cbb6a)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:31:51.524286 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48d3e05-9e1e-45ae-b425-ab40b7f83e6d admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:31:51.574469 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:31:51.635752 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-f48d3e05-9e1e-45ae-b425-ab40b7f83e6d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '307655b70b684fb8bbee7a37c42cbb6a', 'request_id': 'req-f48d3e05-9e1e-45ae-b425-ab40b7f83e6d'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:31:51.644134 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-f48d3e05-9e1e-45ae-b425-ab40b7f83e6d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '307655b70b684fb8bbee7a37c42cbb6a', 'request_id': 'req-f48d3e05-9e1e-45ae-b425-ab40b7f83e6d'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:31:51.657155 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-f48d3e05-9e1e-45ae-b425-ab40b7f83e6d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '307655b70b684fb8bbee7a37c42cbb6a', 'request_id': 'req-f48d3e05-9e1e-45ae-b425-ab40b7f83e6d'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:31:51.679175 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-f48d3e05-9e1e-45ae-b425-ab40b7f83e6d admin admin] The token cache is being invalidate because project 307655b70b684fb8bbee7a37c42cbb6a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80153) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:31:51.679551 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-f48d3e05-9e1e-45ae-b425-ab40b7f83e6d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:31:51.680172 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-f48d3e05-9e1e-45ae-b425-ab40b7f83e6d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:31:51.682410 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1339/2676] 10.222.0.64 () {58 vars in 1233 bytes} [Tue Aug 2 10:31:51 2022] DELETE /identity/v3/projects/307655b70b684fb8bbee7a37c42cbb6a => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:31:51.690882 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701958b6-fb94-4d9b-95c5-c91beedfba53 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:31:51.691480 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701958b6-fb94-4d9b-95c5-c91beedfba53 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:31:51.691480 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701958b6-fb94-4d9b-95c5-c91beedfba53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:31:51.712732 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-701958b6-fb94-4d9b-95c5-c91beedfba53 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:31:51.714107 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-701958b6-fb94-4d9b-95c5-c91beedfba53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:31:51.746821 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:31:51.801601 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1338/2677] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:31:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:31:51.809619 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243020b5-8460-425d-a87a-be2ca863a1e6 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:31:51.809953 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243020b5-8460-425d-a87a-be2ca863a1e6 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:31:51.810294 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243020b5-8460-425d-a87a-be2ca863a1e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:31:51.853123 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-243020b5-8460-425d-a87a-be2ca863a1e6 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:31:51.855489 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-243020b5-8460-425d-a87a-be2ca863a1e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:31:51.866461 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1340/2678] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:31:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:31:51.871929 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701958b6-fb94-4d9b-95c5-c91beedfba53 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:31:51.895977 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701958b6-fb94-4d9b-95c5-c91beedfba53 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:31:51.904530 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22a7ab4-81fe-46e7-9378-c396d9064c1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:31:51.906069 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22a7ab4-81fe-46e7-9378-c396d9064c1e admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:31:51.906342 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22a7ab4-81fe-46e7-9378-c396d9064c1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:31:51.907338 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22a7ab4-81fe-46e7-9378-c396d9064c1e admin admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:31:51.908843 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22a7ab4-81fe-46e7-9378-c396d9064c1e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:31:51.909481 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22a7ab4-81fe-46e7-9378-c396d9064c1e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:31:51.912526 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22a7ab4-81fe-46e7-9378-c396d9064c1e admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:31:51.938317 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1339/2679] 10.222.0.64 () {58 vars in 1153 bytes} [Tue Aug 2 10:31:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:31:51.943064 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243020b5-8460-425d-a87a-be2ca863a1e6 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:31:51.955823 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243020b5-8460-425d-a87a-be2ca863a1e6 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:31:51.962713 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a01f66-a741-4263-84bc-26900e8f7119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:31:51.963532 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a01f66-a741-4263-84bc-26900e8f7119 admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:31:51.963841 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a01f66-a741-4263-84bc-26900e8f7119 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:31:51.964303 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a01f66-a741-4263-84bc-26900e8f7119 admin admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:31:51.965074 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a01f66-a741-4263-84bc-26900e8f7119 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:31:51.968064 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a01f66-a741-4263-84bc-26900e8f7119 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:31:52.005810 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1341/2680] 10.222.0.64 () {60 vars in 1153 bytes} [Tue Aug 2 10:31:51 2022] POST /identity/v3/projects => generated 361 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:31:52.010919 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22a7ab4-81fe-46e7-9378-c396d9064c1e admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:31:52.023444 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22a7ab4-81fe-46e7-9378-c396d9064c1e admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:31:52.108026 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f895d03-55ef-476a-9adc-794a6c953750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:31:52.108743 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f895d03-55ef-476a-9adc-794a6c953750 admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:31:52.109024 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f895d03-55ef-476a-9adc-794a6c953750 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:31:52.109306 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f895d03-55ef-476a-9adc-794a6c953750 admin admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:31:52.110600 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f895d03-55ef-476a-9adc-794a6c953750 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:31:52.112692 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f895d03-55ef-476a-9adc-794a6c953750 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:31:52.137052 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1340/2681] 10.222.0.64 () {60 vars in 1144 bytes} [Tue Aug 2 10:31:52 2022] POST /identity/v3/users => generated 281 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:31:52.142101 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a01f66-a741-4263-84bc-26900e8f7119 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:31:52.155668 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a01f66-a741-4263-84bc-26900e8f7119 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:31:52.161252 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb2b78f-846c-4c23-bc99-6609d28f6abd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:31:52.161946 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb2b78f-846c-4c23-bc99-6609d28f6abd admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:31:52.162234 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb2b78f-846c-4c23-bc99-6609d28f6abd admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:31:52.162531 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb2b78f-846c-4c23-bc99-6609d28f6abd admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:31:52.163225 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb2b78f-846c-4c23-bc99-6609d28f6abd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:31:52.164899 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb2b78f-846c-4c23-bc99-6609d28f6abd admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:31:52.177648 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1342/2682] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:31:52 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:31:52.182155 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f895d03-55ef-476a-9adc-794a6c953750 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:31:52.194152 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f895d03-55ef-476a-9adc-794a6c953750 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:31:52.206646 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62668dc4-db7b-4214-9cb8-5a60d443e5fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:31:52.207320 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62668dc4-db7b-4214-9cb8-5a60d443e5fc admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:31:52.207606 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62668dc4-db7b-4214-9cb8-5a60d443e5fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:31:52.207990 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62668dc4-db7b-4214-9cb8-5a60d443e5fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:31:52.208785 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62668dc4-db7b-4214-9cb8-5a60d443e5fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:31:52.210389 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62668dc4-db7b-4214-9cb8-5a60d443e5fc admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:31:52.227879 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1341/2683] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:31:52 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:31:52.229347 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb2b78f-846c-4c23-bc99-6609d28f6abd admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:31:52.240570 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb2b78f-846c-4c23-bc99-6609d28f6abd admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:31:52.246489 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89df877d-824b-4b83-8829-95a7c61165d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:31:52.247415 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89df877d-824b-4b83-8829-95a7c61165d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:31:52.247864 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89df877d-824b-4b83-8829-95a7c61165d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:31:52.248295 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89df877d-824b-4b83-8829-95a7c61165d3 admin admin] PATH_INFO: `/v3/projects/899de9bdda284ad0ad640ccc58901e2d/users/bcd4f4b3110e4f7998ad122daf15611d/roles/66c1074780954f8e87ed11a5c4e8c36a` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:31:52.267381 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89df877d-824b-4b83-8829-95a7c61165d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=899de9bdda284ad0ad640ccc58901e2d, user_id=bcd4f4b3110e4f7998ad122daf15611d, role_id=66c1074780954f8e87ed11a5c4e8c36a)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:31:52.270381 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89df877d-824b-4b83-8829-95a7c61165d3 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:31:52.291343 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1343/2684] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 10:31:52 2022] PUT /identity/v3/projects/899de9bdda284ad0ad640ccc58901e2d/users/bcd4f4b3110e4f7998ad122daf15611d/roles/66c1074780954f8e87ed11a5c4e8c36a => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:31:52.305146 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6798c2-c305-45ac-9456-fc76228b13fc None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:31:52.305535 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6798c2-c305-45ac-9456-fc76228b13fc None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:31:52.305901 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6798c2-c305-45ac-9456-fc76228b13fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:31:52.343200 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-2b6798c2-c305-45ac-9456-fc76228b13fc None None] MFA Rules not processed for user `bcd4f4b3110e4f7998ad122daf15611d`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:31:52.344941 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-2b6798c2-c305-45ac-9456-fc76228b13fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:31:52.440100 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1342/2685] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:31:52 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:31:53.033071 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89df877d-824b-4b83-8829-95a7c61165d3 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:31:53.035946 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-89df877d-824b-4b83-8829-95a7c61165d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:31:53.117141 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89df877d-824b-4b83-8829-95a7c61165d3 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:31:53.123728 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b169c8-3c56-4cb0-8f7d-2a97a1b77c3d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99655e1f9ca24472a86cd26b0dbd1fc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:31:53.124364 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b169c8-3c56-4cb0-8f7d-2a97a1b77c3d service nova] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:31:53.124649 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b169c8-3c56-4cb0-8f7d-2a97a1b77c3d service nova] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:31:53.124925 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b169c8-3c56-4cb0-8f7d-2a97a1b77c3d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:31:53.127622 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-13b169c8-3c56-4cb0-8f7d-2a97a1b77c3d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:31:53.141069 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b169c8-3c56-4cb0-8f7d-2a97a1b77c3d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:31:53.142768 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b169c8-3c56-4cb0-8f7d-2a97a1b77c3d service nova] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:31:53.154020 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1344/2686] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:31:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:32:02.436058 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=86,keystone:INSERT=3 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:32:03.149464 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=78,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:33:02.050482 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6798c2-c305-45ac-9456-fc76228b13fc None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:33:02.051903 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-2b6798c2-c305-45ac-9456-fc76228b13fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:33:02.140330 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6798c2-c305-45ac-9456-fc76228b13fc None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:33:02.145267 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e72b586-a4df-4029-b09e-f29c8ab08cfa service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b770f073c86b4d88837a8b1f796a8b1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:33:02.145879 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e72b586-a4df-4029-b09e-f29c8ab08cfa service placement] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:33:02.146164 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e72b586-a4df-4029-b09e-f29c8ab08cfa service placement] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:33:02.146439 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e72b586-a4df-4029-b09e-f29c8ab08cfa service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:33:02.148173 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-9e72b586-a4df-4029-b09e-f29c8ab08cfa service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:33:02.159358 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e72b586-a4df-4029-b09e-f29c8ab08cfa service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:33:02.160977 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e72b586-a4df-4029-b09e-f29c8ab08cfa service placement] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:33:02.171162 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1343/2687] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:33:02 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:33:12.166511 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=41 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:33:49.701842 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b169c8-3c56-4cb0-8f7d-2a97a1b77c3d service nova] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:33:49.703472 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-13b169c8-3c56-4cb0-8f7d-2a97a1b77c3d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:33:49.818062 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b169c8-3c56-4cb0-8f7d-2a97a1b77c3d service nova] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:33:49.822749 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aede187-d00f-4538-ba1e-f2e09f261369 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d7de62b9ee4039bdc79627ac7fb499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:33:49.823490 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aede187-d00f-4538-ba1e-f2e09f261369 service cinder] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:33:49.823819 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aede187-d00f-4538-ba1e-f2e09f261369 service cinder] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:33:49.824137 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aede187-d00f-4538-ba1e-f2e09f261369 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:33:49.826406 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-8aede187-d00f-4538-ba1e-f2e09f261369 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:33:49.942202 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aede187-d00f-4538-ba1e-f2e09f261369 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:33:49.945076 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aede187-d00f-4538-ba1e-f2e09f261369 service cinder] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:33:49.977697 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1345/2688] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:33:49 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:33:59.974073 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=70 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:34:48.441874 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e72b586-a4df-4029-b09e-f29c8ab08cfa service placement] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:48.443721 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-9e72b586-a4df-4029-b09e-f29c8ab08cfa service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:34:48.476592 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e72b586-a4df-4029-b09e-f29c8ab08cfa service placement] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:48.488525 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733cad9a-c3dd-4199-87ff-7f877ab33f59 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99655e1f9ca24472a86cd26b0dbd1fc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:48.489537 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733cad9a-c3dd-4199-87ff-7f877ab33f59 service nova] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:48.490039 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733cad9a-c3dd-4199-87ff-7f877ab33f59 service nova] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:48.490647 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733cad9a-c3dd-4199-87ff-7f877ab33f59 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:48.493622 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-733cad9a-c3dd-4199-87ff-7f877ab33f59 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:34:48.638730 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733cad9a-c3dd-4199-87ff-7f877ab33f59 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:48.641763 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733cad9a-c3dd-4199-87ff-7f877ab33f59 service nova] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:48.673761 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1344/2689] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:34:48 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:34:58.668739 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=41 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:50.179244 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aede187-d00f-4538-ba1e-f2e09f261369 service cinder] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:50.188478 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aede187-d00f-4538-ba1e-f2e09f261369 service cinder] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:50.192439 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5122bc-d12c-4e2f-bc75-25b2db4beada service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d7de62b9ee4039bdc79627ac7fb499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:50.193223 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5122bc-d12c-4e2f-bc75-25b2db4beada service cinder] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:50.193510 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5122bc-d12c-4e2f-bc75-25b2db4beada service cinder] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:50.193786 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5122bc-d12c-4e2f-bc75-25b2db4beada service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:50.195782 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-3d5122bc-d12c-4e2f-bc75-25b2db4beada service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:35:50.276111 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733cad9a-c3dd-4199-87ff-7f877ab33f59 service nova] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:50.292932 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733cad9a-c3dd-4199-87ff-7f877ab33f59 service nova] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:50.299942 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4c25ce-f769-423a-b80c-1752e5cf205b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99655e1f9ca24472a86cd26b0dbd1fc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:50.301030 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4c25ce-f769-423a-b80c-1752e5cf205b service nova] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:50.301541 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4c25ce-f769-423a-b80c-1752e5cf205b service nova] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:50.301817 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4c25ce-f769-423a-b80c-1752e5cf205b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:50.303661 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-cb4c25ce-f769-423a-b80c-1752e5cf205b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:35:50.317900 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5122bc-d12c-4e2f-bc75-25b2db4beada service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:50.319486 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5122bc-d12c-4e2f-bc75-25b2db4beada service cinder] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:50.330275 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1346/2690] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:35:50 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:35:50.331222 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5122bc-d12c-4e2f-bc75-25b2db4beada service cinder] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:50.332587 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-3d5122bc-d12c-4e2f-bc75-25b2db4beada service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:35:50.346279 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5122bc-d12c-4e2f-bc75-25b2db4beada service cinder] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:50.350714 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb6f710-59f6-4325-af95-62accf8df23f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d7de62b9ee4039bdc79627ac7fb499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:50.352318 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb6f710-59f6-4325-af95-62accf8df23f service cinder] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:50.352888 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb6f710-59f6-4325-af95-62accf8df23f service cinder] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:50.353225 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb6f710-59f6-4325-af95-62accf8df23f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:50.356158 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-0bb6f710-59f6-4325-af95-62accf8df23f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:35:50.373432 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4c25ce-f769-423a-b80c-1752e5cf205b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:50.375074 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4c25ce-f769-423a-b80c-1752e5cf205b service nova] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:50.385098 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1345/2691] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:35:50 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:35:50.388068 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb6f710-59f6-4325-af95-62accf8df23f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:50.389583 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb6f710-59f6-4325-af95-62accf8df23f service cinder] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:50.401320 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1347/2692] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:35:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:36:00.380078 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=28 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:00.396674 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=44 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:51.520284 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4c25ce-f769-423a-b80c-1752e5cf205b service nova] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:51.535154 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4c25ce-f769-423a-b80c-1752e5cf205b service nova] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:51.543218 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e7c8ce-7eb7-4dbd-a8d2-4dba979ef800 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99655e1f9ca24472a86cd26b0dbd1fc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:51.544087 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e7c8ce-7eb7-4dbd-a8d2-4dba979ef800 service nova] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:51.544422 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e7c8ce-7eb7-4dbd-a8d2-4dba979ef800 service nova] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:51.544721 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e7c8ce-7eb7-4dbd-a8d2-4dba979ef800 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:51.546803 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-d5e7c8ce-7eb7-4dbd-a8d2-4dba979ef800 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:36:51.658202 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e7c8ce-7eb7-4dbd-a8d2-4dba979ef800 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:51.661164 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e7c8ce-7eb7-4dbd-a8d2-4dba979ef800 service nova] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:51.677786 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1346/2693] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:36:51 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:37:01.670811 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=28 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:37:50.372234 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb6f710-59f6-4325-af95-62accf8df23f service cinder] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:37:50.381253 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb6f710-59f6-4325-af95-62accf8df23f service cinder] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:37:50.384883 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd7df37-926a-4757-9cd5-f71c751bc095 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d7de62b9ee4039bdc79627ac7fb499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:37:50.385501 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd7df37-926a-4757-9cd5-f71c751bc095 service cinder] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:50.385789 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd7df37-926a-4757-9cd5-f71c751bc095 service cinder] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:50.386064 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd7df37-926a-4757-9cd5-f71c751bc095 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:50.394539 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd7df37-926a-4757-9cd5-f71c751bc095 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:37:50.396500 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd7df37-926a-4757-9cd5-f71c751bc095 service cinder] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:37:50.405702 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1348/2694] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:37:50 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:37:50.522339 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e7c8ce-7eb7-4dbd-a8d2-4dba979ef800 service nova] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:37:50.532699 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e7c8ce-7eb7-4dbd-a8d2-4dba979ef800 service nova] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:37:50.537378 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ed4b47-92a6-4362-b8df-df5e91931080 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d7de62b9ee4039bdc79627ac7fb499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:37:50.538244 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ed4b47-92a6-4362-b8df-df5e91931080 service cinder] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:50.538578 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ed4b47-92a6-4362-b8df-df5e91931080 service cinder] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:50.538962 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ed4b47-92a6-4362-b8df-df5e91931080 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:50.541648 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-66ed4b47-92a6-4362-b8df-df5e91931080 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:37:50.560207 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ed4b47-92a6-4362-b8df-df5e91931080 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:37:50.561859 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ed4b47-92a6-4362-b8df-df5e91931080 service cinder] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:37:50.575263 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1347/2695] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:37:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:37:50.888473 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd7df37-926a-4757-9cd5-f71c751bc095 service cinder] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:37:50.977565 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd7df37-926a-4757-9cd5-f71c751bc095 service cinder] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:37:50.982256 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16092057-5d85-46d8-bb82-4ca41c639782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:37:50.982911 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16092057-5d85-46d8-bb82-4ca41c639782 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:50.983201 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16092057-5d85-46d8-bb82-4ca41c639782 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:50.983485 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16092057-5d85-46d8-bb82-4ca41c639782 admin admin] PATH_INFO: `/v3/users/bcd4f4b3110e4f7998ad122daf15611d` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:50.984677 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16092057-5d85-46d8-bb82-4ca41c639782 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bcd4f4b3110e4f7998ad122daf15611d)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:37:50.986589 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16092057-5d85-46d8-bb82-4ca41c639782 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:37:51.033644 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-16092057-5d85-46d8-bb82-4ca41c639782 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bcd4f4b3110e4f7998ad122daf15611d'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:37:51.040536 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-16092057-5d85-46d8-bb82-4ca41c639782 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bcd4f4b3110e4f7998ad122daf15611d'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:37:51.047959 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-16092057-5d85-46d8-bb82-4ca41c639782 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bcd4f4b3110e4f7998ad122daf15611d'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:37:51.048607 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-16092057-5d85-46d8-bb82-4ca41c639782 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bcd4f4b3110e4f7998ad122daf15611d'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:37:51.049308 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-16092057-5d85-46d8-bb82-4ca41c639782 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bcd4f4b3110e4f7998ad122daf15611d'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:37:51.066608 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1349/2696] 10.222.0.64 () {58 vars in 1224 bytes} [Tue Aug 2 10:37:50 2022] DELETE /identity/v3/users/bcd4f4b3110e4f7998ad122daf15611d => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:37:51.071769 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18075663-3e99-4a5d-913b-d36f07ade053 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:51.072064 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18075663-3e99-4a5d-913b-d36f07ade053 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:51.072399 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18075663-3e99-4a5d-913b-d36f07ade053 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:51.111581 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-18075663-3e99-4a5d-913b-d36f07ade053 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:37:51.112824 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-18075663-3e99-4a5d-913b-d36f07ade053 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:37:51.172294 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1348/2697] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:37:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:37:51.183057 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16092057-5d85-46d8-bb82-4ca41c639782 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:37:51.184767 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-16092057-5d85-46d8-bb82-4ca41c639782 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:37:51.237485 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:37:51.256594 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16092057-5d85-46d8-bb82-4ca41c639782 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:37:51.260276 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649a7ee5-bcf8-45f3-9002-dbc2d62df049 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:37:51.260917 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649a7ee5-bcf8-45f3-9002-dbc2d62df049 service neutron] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:51.261206 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649a7ee5-bcf8-45f3-9002-dbc2d62df049 service neutron] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:51.261485 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649a7ee5-bcf8-45f3-9002-dbc2d62df049 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:51.270600 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649a7ee5-bcf8-45f3-9002-dbc2d62df049 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:37:51.272041 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649a7ee5-bcf8-45f3-9002-dbc2d62df049 service neutron] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:37:51.281577 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1350/2698] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:37:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:37:51.569336 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18075663-3e99-4a5d-913b-d36f07ade053 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:37:51.570755 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-18075663-3e99-4a5d-913b-d36f07ade053 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:37:51.582540 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18075663-3e99-4a5d-913b-d36f07ade053 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:37:51.587604 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03131cf-d229-4a8c-8d93-cb7db25e5494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:37:51.589200 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03131cf-d229-4a8c-8d93-cb7db25e5494 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:51.589492 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03131cf-d229-4a8c-8d93-cb7db25e5494 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:51.590105 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03131cf-d229-4a8c-8d93-cb7db25e5494 admin admin] PATH_INFO: `/v3/projects/899de9bdda284ad0ad640ccc58901e2d` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:51.591302 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03131cf-d229-4a8c-8d93-cb7db25e5494 admin admin] RBAC: Authorizing `identity:delete_project(project_id=899de9bdda284ad0ad640ccc58901e2d)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:37:51.594103 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03131cf-d229-4a8c-8d93-cb7db25e5494 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:37:51.643114 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-d03131cf-d229-4a8c-8d93-cb7db25e5494 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '899de9bdda284ad0ad640ccc58901e2d', 'request_id': 'req-d03131cf-d229-4a8c-8d93-cb7db25e5494'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:37:51.649292 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-d03131cf-d229-4a8c-8d93-cb7db25e5494 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '899de9bdda284ad0ad640ccc58901e2d', 'request_id': 'req-d03131cf-d229-4a8c-8d93-cb7db25e5494'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:37:51.655513 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-d03131cf-d229-4a8c-8d93-cb7db25e5494 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '899de9bdda284ad0ad640ccc58901e2d', 'request_id': 'req-d03131cf-d229-4a8c-8d93-cb7db25e5494'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:37:51.669565 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-d03131cf-d229-4a8c-8d93-cb7db25e5494 admin admin] The token cache is being invalidate because project 899de9bdda284ad0ad640ccc58901e2d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80154) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:37:51.669861 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-d03131cf-d229-4a8c-8d93-cb7db25e5494 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:37:51.670489 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-d03131cf-d229-4a8c-8d93-cb7db25e5494 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:37:51.672013 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1349/2699] 10.222.0.64 () {58 vars in 1233 bytes} [Tue Aug 2 10:37:51 2022] DELETE /identity/v3/projects/899de9bdda284ad0ad640ccc58901e2d => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:37:51.677726 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3bf498-a395-4622-993e-fd1f3ac26b7f None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:51.678219 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3bf498-a395-4622-993e-fd1f3ac26b7f None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:51.678219 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3bf498-a395-4622-993e-fd1f3ac26b7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:51.733342 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-fe3bf498-a395-4622-993e-fd1f3ac26b7f None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:37:51.734568 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-fe3bf498-a395-4622-993e-fd1f3ac26b7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:37:51.800672 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1351/2700] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:37:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:37:51.810861 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3ad694-814c-4ed4-9036-e15e35d7920b None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:51.811157 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3ad694-814c-4ed4-9036-e15e35d7920b None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:51.811445 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3ad694-814c-4ed4-9036-e15e35d7920b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:51.832405 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-6f3ad694-814c-4ed4-9036-e15e35d7920b None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:37:51.833665 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-6f3ad694-814c-4ed4-9036-e15e35d7920b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:37:51.842254 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1350/2701] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:37:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:37:51.846504 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3bf498-a395-4622-993e-fd1f3ac26b7f None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:37:51.857673 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3bf498-a395-4622-993e-fd1f3ac26b7f None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:37:51.862508 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a3525f-86a7-471f-8331-2af5537e0e5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:37:51.863144 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a3525f-86a7-471f-8331-2af5537e0e5d admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:51.863436 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a3525f-86a7-471f-8331-2af5537e0e5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:51.863716 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a3525f-86a7-471f-8331-2af5537e0e5d admin admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:51.865052 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a3525f-86a7-471f-8331-2af5537e0e5d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:37:51.865376 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a3525f-86a7-471f-8331-2af5537e0e5d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:37:51.867033 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a3525f-86a7-471f-8331-2af5537e0e5d admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:37:51.880573 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1352/2702] 10.222.0.64 () {58 vars in 1153 bytes} [Tue Aug 2 10:37:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:37:51.884942 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3ad694-814c-4ed4-9036-e15e35d7920b None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:37:51.895010 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3ad694-814c-4ed4-9036-e15e35d7920b None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:37:51.900578 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d06c910-99c9-41de-9655-5b930040147b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:37:51.901246 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d06c910-99c9-41de-9655-5b930040147b admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:51.901532 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d06c910-99c9-41de-9655-5b930040147b admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:51.901807 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d06c910-99c9-41de-9655-5b930040147b admin admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:51.902499 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d06c910-99c9-41de-9655-5b930040147b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:37:51.904489 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d06c910-99c9-41de-9655-5b930040147b admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:37:51.946073 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1351/2703] 10.222.0.64 () {60 vars in 1153 bytes} [Tue Aug 2 10:37:51 2022] POST /identity/v3/projects => generated 365 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:37:51.950270 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a3525f-86a7-471f-8331-2af5537e0e5d admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:37:51.961498 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a3525f-86a7-471f-8331-2af5537e0e5d admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:37:52.027457 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd24f9f-83d3-4315-a0d9-5d8a4dabcfbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:37:52.028126 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd24f9f-83d3-4315-a0d9-5d8a4dabcfbf admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:52.028444 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd24f9f-83d3-4315-a0d9-5d8a4dabcfbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:52.028808 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd24f9f-83d3-4315-a0d9-5d8a4dabcfbf admin admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:52.029508 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd24f9f-83d3-4315-a0d9-5d8a4dabcfbf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:37:52.031504 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd24f9f-83d3-4315-a0d9-5d8a4dabcfbf admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:37:52.056221 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1353/2704] 10.222.0.64 () {60 vars in 1144 bytes} [Tue Aug 2 10:37:51 2022] POST /identity/v3/users => generated 283 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:37:52.060696 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d06c910-99c9-41de-9655-5b930040147b admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:37:52.071936 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d06c910-99c9-41de-9655-5b930040147b admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:37:52.076978 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce918139-d4d2-4ad4-8618-d3e75c407eaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:37:52.077682 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce918139-d4d2-4ad4-8618-d3e75c407eaa admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:52.077973 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce918139-d4d2-4ad4-8618-d3e75c407eaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:52.078256 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce918139-d4d2-4ad4-8618-d3e75c407eaa admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:52.078983 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce918139-d4d2-4ad4-8618-d3e75c407eaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:37:52.080659 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce918139-d4d2-4ad4-8618-d3e75c407eaa admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:37:52.092270 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1352/2705] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:37:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:37:52.096401 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd24f9f-83d3-4315-a0d9-5d8a4dabcfbf admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:37:52.107280 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd24f9f-83d3-4315-a0d9-5d8a4dabcfbf admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:37:52.112809 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6e9f3e-1ad4-4178-a4bd-c9fffaf53488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:37:52.113460 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6e9f3e-1ad4-4178-a4bd-c9fffaf53488 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:52.113748 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6e9f3e-1ad4-4178-a4bd-c9fffaf53488 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:52.114028 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6e9f3e-1ad4-4178-a4bd-c9fffaf53488 admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:52.115305 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6e9f3e-1ad4-4178-a4bd-c9fffaf53488 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:37:52.117031 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6e9f3e-1ad4-4178-a4bd-c9fffaf53488 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:37:52.128849 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1354/2706] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:37:52 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:37:52.133575 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce918139-d4d2-4ad4-8618-d3e75c407eaa admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:37:52.140052 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:37:52.144238 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce918139-d4d2-4ad4-8618-d3e75c407eaa admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:37:52.150591 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd823ec3-01a4-483f-9e58-c9241c0467e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:37:52.151391 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd823ec3-01a4-483f-9e58-c9241c0467e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:52.151955 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd823ec3-01a4-483f-9e58-c9241c0467e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:52.152616 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd823ec3-01a4-483f-9e58-c9241c0467e1 admin admin] PATH_INFO: `/v3/projects/5175365a2db9489986f5a84961a5d0fb/users/ae5cca45c480411cb12758b3785cf1af/roles/66c1074780954f8e87ed11a5c4e8c36a` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:52.171157 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd823ec3-01a4-483f-9e58-c9241c0467e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5175365a2db9489986f5a84961a5d0fb, user_id=ae5cca45c480411cb12758b3785cf1af, role_id=66c1074780954f8e87ed11a5c4e8c36a)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:37:52.173907 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd823ec3-01a4-483f-9e58-c9241c0467e1 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:37:52.191837 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1353/2707] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 10:37:52 2022] PUT /identity/v3/projects/5175365a2db9489986f5a84961a5d0fb/users/ae5cca45c480411cb12758b3785cf1af/roles/66c1074780954f8e87ed11a5c4e8c36a => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:37:52.204724 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293ed6f1-2cae-4974-9b3a-835b567595a2 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:52.204945 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293ed6f1-2cae-4974-9b3a-835b567595a2 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:52.205180 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293ed6f1-2cae-4974-9b3a-835b567595a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:52.235414 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-293ed6f1-2cae-4974-9b3a-835b567595a2 None None] MFA Rules not processed for user `ae5cca45c480411cb12758b3785cf1af`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:37:52.236778 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-293ed6f1-2cae-4974-9b3a-835b567595a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:37:52.248824 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:37:52.316414 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1355/2708] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:37:52 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:38:02.185910 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=8,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:38:02.313226 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=22 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:38:09.051367 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd823ec3-01a4-483f-9e58-c9241c0467e1 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:38:09.053220 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-dd823ec3-01a4-483f-9e58-c9241c0467e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:38:09.151215 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd823ec3-01a4-483f-9e58-c9241c0467e1 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:38:09.156360 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04921610-ea7c-476c-809e-3c95e8c7cdcf service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b770f073c86b4d88837a8b1f796a8b1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:38:09.157019 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04921610-ea7c-476c-809e-3c95e8c7cdcf service placement] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:38:09.157307 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04921610-ea7c-476c-809e-3c95e8c7cdcf service placement] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:38:09.157586 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04921610-ea7c-476c-809e-3c95e8c7cdcf service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:38:09.159574 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-04921610-ea7c-476c-809e-3c95e8c7cdcf service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:38:09.171765 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04921610-ea7c-476c-809e-3c95e8c7cdcf service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:38:09.173381 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04921610-ea7c-476c-809e-3c95e8c7cdcf service placement] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:38:09.184168 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1354/2709] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:38:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:38:19.179714 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=43 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:39:50.848480 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293ed6f1-2cae-4974-9b3a-835b567595a2 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:39:50.849921 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-293ed6f1-2cae-4974-9b3a-835b567595a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:39:50.929728 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293ed6f1-2cae-4974-9b3a-835b567595a2 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:39:50.934097 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543fba3a-4b4a-4b31-b6a4-379b0de87d64 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d7de62b9ee4039bdc79627ac7fb499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:39:50.935205 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543fba3a-4b4a-4b31-b6a4-379b0de87d64 service cinder] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:39:50.935558 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543fba3a-4b4a-4b31-b6a4-379b0de87d64 service cinder] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:39:50.935838 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543fba3a-4b4a-4b31-b6a4-379b0de87d64 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:39:50.937869 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-543fba3a-4b4a-4b31-b6a4-379b0de87d64 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:39:50.959468 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04921610-ea7c-476c-809e-3c95e8c7cdcf service placement] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:39:50.961094 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-04921610-ea7c-476c-809e-3c95e8c7cdcf service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:39:50.983279 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04921610-ea7c-476c-809e-3c95e8c7cdcf service placement] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:39:50.988030 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8b7f3e-edd2-4347-8c79-c22e90b06c20 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d7de62b9ee4039bdc79627ac7fb499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:39:50.988740 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8b7f3e-edd2-4347-8c79-c22e90b06c20 service cinder] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:39:50.989055 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8b7f3e-edd2-4347-8c79-c22e90b06c20 service cinder] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:39:50.989366 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8b7f3e-edd2-4347-8c79-c22e90b06c20 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:39:51.011879 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543fba3a-4b4a-4b31-b6a4-379b0de87d64 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:39:51.013378 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543fba3a-4b4a-4b31-b6a4-379b0de87d64 service cinder] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:39:51.014814 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8b7f3e-edd2-4347-8c79-c22e90b06c20 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:39:51.017012 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8b7f3e-edd2-4347-8c79-c22e90b06c20 service cinder] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:39:51.023574 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1356/2710] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:39:50 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:39:51.026912 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1355/2711] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:39:50 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:39:51.075481 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543fba3a-4b4a-4b31-b6a4-379b0de87d64 service cinder] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:39:51.086045 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543fba3a-4b4a-4b31-b6a4-379b0de87d64 service cinder] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:39:51.090524 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f565c4dd-5547-451d-bd29-f2d620e2c2e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d7de62b9ee4039bdc79627ac7fb499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:39:51.091274 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f565c4dd-5547-451d-bd29-f2d620e2c2e5 service cinder] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:39:51.091592 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f565c4dd-5547-451d-bd29-f2d620e2c2e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:39:51.091883 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f565c4dd-5547-451d-bd29-f2d620e2c2e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:39:51.093822 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-f565c4dd-5547-451d-bd29-f2d620e2c2e5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:39:51.164737 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f565c4dd-5547-451d-bd29-f2d620e2c2e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:39:51.166205 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f565c4dd-5547-451d-bd29-f2d620e2c2e5 service cinder] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:39:51.177920 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1357/2712] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:39:51 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:40:01.022548 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=17 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:40:01.173030 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=88 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:41:51.466513 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8b7f3e-edd2-4347-8c79-c22e90b06c20 service cinder] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:41:51.475448 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8b7f3e-edd2-4347-8c79-c22e90b06c20 service cinder] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:41:51.479233 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dff7c52-e266-48d8-b36a-2ad3e2c23d6b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d7de62b9ee4039bdc79627ac7fb499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:41:51.479850 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dff7c52-e266-48d8-b36a-2ad3e2c23d6b service cinder] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:41:51.480136 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dff7c52-e266-48d8-b36a-2ad3e2c23d6b service cinder] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:41:51.480444 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dff7c52-e266-48d8-b36a-2ad3e2c23d6b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:41:51.482730 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-7dff7c52-e266-48d8-b36a-2ad3e2c23d6b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:41:51.563387 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dff7c52-e266-48d8-b36a-2ad3e2c23d6b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:41:51.564972 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dff7c52-e266-48d8-b36a-2ad3e2c23d6b service cinder] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:41:51.585901 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1356/2713] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:41:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:41:51.593318 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f565c4dd-5547-451d-bd29-f2d620e2c2e5 service cinder] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:41:51.603037 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f565c4dd-5547-451d-bd29-f2d620e2c2e5 service cinder] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:41:51.607339 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6faee7-9a8c-4c4d-a0df-e94ac976248e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d7de62b9ee4039bdc79627ac7fb499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:41:51.607976 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6faee7-9a8c-4c4d-a0df-e94ac976248e service cinder] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:41:51.608308 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6faee7-9a8c-4c4d-a0df-e94ac976248e service cinder] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:41:51.608590 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6faee7-9a8c-4c4d-a0df-e94ac976248e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:41:51.610503 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-6e6faee7-9a8c-4c4d-a0df-e94ac976248e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:41:51.699676 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6faee7-9a8c-4c4d-a0df-e94ac976248e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:41:51.701207 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6faee7-9a8c-4c4d-a0df-e94ac976248e service cinder] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:41:51.721390 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1358/2714] 10.222.0.64 () {60 vars in 1396 bytes} [Tue Aug 2 10:41:51 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3250 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:41:51.807092 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dff7c52-e266-48d8-b36a-2ad3e2c23d6b service cinder] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:41:51.818470 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dff7c52-e266-48d8-b36a-2ad3e2c23d6b service cinder] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:41:51.822420 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460af884-518e-4f01-b31d-4ccf0cb61548 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d7de62b9ee4039bdc79627ac7fb499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:41:51.823298 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460af884-518e-4f01-b31d-4ccf0cb61548 service cinder] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:41:51.823298 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460af884-518e-4f01-b31d-4ccf0cb61548 service cinder] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:41:51.823780 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460af884-518e-4f01-b31d-4ccf0cb61548 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:41:51.826061 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-460af884-518e-4f01-b31d-4ccf0cb61548 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:41:51.912338 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460af884-518e-4f01-b31d-4ccf0cb61548 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:41:51.914111 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460af884-518e-4f01-b31d-4ccf0cb61548 service cinder] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:41:51.937274 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1357/2715] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:41:51 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:42:01.718617 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=41 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:42:01.933418 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=77 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:43:10.028870 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6faee7-9a8c-4c4d-a0df-e94ac976248e service cinder] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:43:10.030280 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-6e6faee7-9a8c-4c4d-a0df-e94ac976248e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:43:10.059879 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6faee7-9a8c-4c4d-a0df-e94ac976248e service cinder] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:43:10.065149 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbeae1e9-ced5-49b2-895a-c313558cffcc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b770f073c86b4d88837a8b1f796a8b1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:43:10.065776 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeae1e9-ced5-49b2-895a-c313558cffcc service placement] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:43:10.066063 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeae1e9-ced5-49b2-895a-c313558cffcc service placement] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:43:10.066336 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeae1e9-ced5-49b2-895a-c313558cffcc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:43:10.075373 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbeae1e9-ced5-49b2-895a-c313558cffcc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:43:10.077007 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbeae1e9-ced5-49b2-895a-c313558cffcc service placement] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:43:10.087257 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1359/2716] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:43:10 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:43:20.082870 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=11 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:43:45.998821 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460af884-518e-4f01-b31d-4ccf0cb61548 service cinder] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:43:46.010602 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460af884-518e-4f01-b31d-4ccf0cb61548 service cinder] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:43:46.015102 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dfef96-0a1b-4d65-b8f5-503f0107ad03 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d7de62b9ee4039bdc79627ac7fb499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:43:46.015808 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dfef96-0a1b-4d65-b8f5-503f0107ad03 service cinder] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:43:46.016165 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dfef96-0a1b-4d65-b8f5-503f0107ad03 service cinder] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:43:46.016547 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dfef96-0a1b-4d65-b8f5-503f0107ad03 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:43:46.018501 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-24dfef96-0a1b-4d65-b8f5-503f0107ad03 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:43:46.032620 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dfef96-0a1b-4d65-b8f5-503f0107ad03 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:43:46.034053 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dfef96-0a1b-4d65-b8f5-503f0107ad03 service cinder] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:43:46.047744 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1358/2717] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:43:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:43:48.334796 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbeae1e9-ced5-49b2-895a-c313558cffcc service placement] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:43:48.334796 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-fbeae1e9-ced5-49b2-895a-c313558cffcc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:43:48.397982 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbeae1e9-ced5-49b2-895a-c313558cffcc service placement] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:43:48.403223 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b44291-116a-4fa1-bf4e-ace2896fcc8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:43:48.403860 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b44291-116a-4fa1-bf4e-ace2896fcc8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:43:48.404149 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b44291-116a-4fa1-bf4e-ace2896fcc8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:43:48.404574 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b44291-116a-4fa1-bf4e-ace2896fcc8d admin admin] PATH_INFO: `/v3/users/e620012d54c948e69764fda7e76238ac` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:43:48.405753 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b44291-116a-4fa1-bf4e-ace2896fcc8d admin admin] RBAC: Authorizing `identity:delete_user(user_id=e620012d54c948e69764fda7e76238ac)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:43:48.407685 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b44291-116a-4fa1-bf4e-ace2896fcc8d admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:43:48.457868 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-b0b44291-116a-4fa1-bf4e-ace2896fcc8d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e620012d54c948e69764fda7e76238ac'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:43:48.465813 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-b0b44291-116a-4fa1-bf4e-ace2896fcc8d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e620012d54c948e69764fda7e76238ac'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:43:48.475454 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-b0b44291-116a-4fa1-bf4e-ace2896fcc8d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e620012d54c948e69764fda7e76238ac'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:43:48.476553 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-b0b44291-116a-4fa1-bf4e-ace2896fcc8d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e620012d54c948e69764fda7e76238ac'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:43:48.477179 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-b0b44291-116a-4fa1-bf4e-ace2896fcc8d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e620012d54c948e69764fda7e76238ac'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:43:48.508180 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1360/2718] 10.222.0.64 () {58 vars in 1224 bytes} [Tue Aug 2 10:43:48 2022] DELETE /identity/v3/users/e620012d54c948e69764fda7e76238ac => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:43:48.514383 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960ebf48-ca30-492a-a33b-8b4de79bc864 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:43:48.514967 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960ebf48-ca30-492a-a33b-8b4de79bc864 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:43:48.515119 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960ebf48-ca30-492a-a33b-8b4de79bc864 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:43:48.533858 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-960ebf48-ca30-492a-a33b-8b4de79bc864 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:43:48.535069 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-960ebf48-ca30-492a-a33b-8b4de79bc864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:43:48.544279 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b44291-116a-4fa1-bf4e-ace2896fcc8d admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:43:48.546201 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-b0b44291-116a-4fa1-bf4e-ace2896fcc8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:43:48.609430 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1359/2719] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:43:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:43:48.620129 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960ebf48-ca30-492a-a33b-8b4de79bc864 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:43:48.621511 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-960ebf48-ca30-492a-a33b-8b4de79bc864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:43:48.627484 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b44291-116a-4fa1-bf4e-ace2896fcc8d admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:43:48.631182 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b1b8a3-4dae-42cf-b209-019136290f42 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:43:48.631928 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b1b8a3-4dae-42cf-b209-019136290f42 service neutron] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:43:48.632286 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b1b8a3-4dae-42cf-b209-019136290f42 service neutron] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:43:48.632568 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b1b8a3-4dae-42cf-b209-019136290f42 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:43:48.633175 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960ebf48-ca30-492a-a33b-8b4de79bc864 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:43:48.634353 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-64b1b8a3-4dae-42cf-b209-019136290f42 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:43:48.636790 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28ad88a-1fe0-4c4c-8a4a-ad48036d18f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:43:48.637398 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28ad88a-1fe0-4c4c-8a4a-ad48036d18f2 service neutron] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:43:48.637685 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28ad88a-1fe0-4c4c-8a4a-ad48036d18f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:43:48.637969 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28ad88a-1fe0-4c4c-8a4a-ad48036d18f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:43:48.646530 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b1b8a3-4dae-42cf-b209-019136290f42 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:43:48.646972 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28ad88a-1fe0-4c4c-8a4a-ad48036d18f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:43:48.648143 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b1b8a3-4dae-42cf-b209-019136290f42 service neutron] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:43:48.648446 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28ad88a-1fe0-4c4c-8a4a-ad48036d18f2 service neutron] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:43:48.658466 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1361/2720] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:43:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:43:48.658828 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1360/2721] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:43:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:43:49.044478 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b1b8a3-4dae-42cf-b209-019136290f42 service neutron] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:43:49.046051 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-64b1b8a3-4dae-42cf-b209-019136290f42 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:43:49.061831 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b1b8a3-4dae-42cf-b209-019136290f42 service neutron] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:43:49.068427 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a37d172-c906-4104-aa77-d6ceaed7fca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:43:49.069829 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a37d172-c906-4104-aa77-d6ceaed7fca4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:43:49.070226 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a37d172-c906-4104-aa77-d6ceaed7fca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:43:49.070554 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a37d172-c906-4104-aa77-d6ceaed7fca4 admin admin] PATH_INFO: `/v3/projects/98140239bedd487c9221e2252bbd0b6a` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:43:49.072339 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a37d172-c906-4104-aa77-d6ceaed7fca4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=98140239bedd487c9221e2252bbd0b6a)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:43:49.074315 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a37d172-c906-4104-aa77-d6ceaed7fca4 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:43:49.085202 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:43:49.134834 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-7a37d172-c906-4104-aa77-d6ceaed7fca4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98140239bedd487c9221e2252bbd0b6a', 'request_id': 'req-7a37d172-c906-4104-aa77-d6ceaed7fca4'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:43:49.140424 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-7a37d172-c906-4104-aa77-d6ceaed7fca4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98140239bedd487c9221e2252bbd0b6a', 'request_id': 'req-7a37d172-c906-4104-aa77-d6ceaed7fca4'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:43:49.147189 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-7a37d172-c906-4104-aa77-d6ceaed7fca4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98140239bedd487c9221e2252bbd0b6a', 'request_id': 'req-7a37d172-c906-4104-aa77-d6ceaed7fca4'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:43:49.160675 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-7a37d172-c906-4104-aa77-d6ceaed7fca4 admin admin] The token cache is being invalidate because project 98140239bedd487c9221e2252bbd0b6a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80153) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:43:49.161055 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-7a37d172-c906-4104-aa77-d6ceaed7fca4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:43:49.161772 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-7a37d172-c906-4104-aa77-d6ceaed7fca4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:43:49.167554 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1362/2722] 10.222.0.64 () {58 vars in 1233 bytes} [Tue Aug 2 10:43:49 2022] DELETE /identity/v3/projects/98140239bedd487c9221e2252bbd0b6a => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:43:49.169547 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a963cc-1e44-4a60-b2ec-7e2a93e4bb94 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:43:49.170034 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a963cc-1e44-4a60-b2ec-7e2a93e4bb94 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:43:49.170369 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a963cc-1e44-4a60-b2ec-7e2a93e4bb94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:43:49.170889 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a37d172-c906-4104-aa77-d6ceaed7fca4 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:43:49.172555 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-7a37d172-c906-4104-aa77-d6ceaed7fca4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:43:49.190917 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-65a963cc-1e44-4a60-b2ec-7e2a93e4bb94 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:43:49.192143 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-65a963cc-1e44-4a60-b2ec-7e2a93e4bb94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:43:49.262254 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1361/2723] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:43:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:43:49.263319 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a963cc-1e44-4a60-b2ec-7e2a93e4bb94 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:43:49.264637 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-65a963cc-1e44-4a60-b2ec-7e2a93e4bb94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:43:49.268693 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a37d172-c906-4104-aa77-d6ceaed7fca4 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:43:49.275795 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e8bd96-8522-44fc-8e50-53b0d0ff1e69 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b770f073c86b4d88837a8b1f796a8b1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:43:49.276557 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e8bd96-8522-44fc-8e50-53b0d0ff1e69 service placement] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:43:49.276906 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e8bd96-8522-44fc-8e50-53b0d0ff1e69 service placement] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:43:49.277179 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a963cc-1e44-4a60-b2ec-7e2a93e4bb94 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:43:49.277371 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e8bd96-8522-44fc-8e50-53b0d0ff1e69 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:43:49.279161 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-62e8bd96-8522-44fc-8e50-53b0d0ff1e69 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:43:49.281889 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c4737d-f4c6-4721-9372-f417aba0bb51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:43:49.282513 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c4737d-f4c6-4721-9372-f417aba0bb51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:43:49.282798 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c4737d-f4c6-4721-9372-f417aba0bb51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:43:49.283074 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c4737d-f4c6-4721-9372-f417aba0bb51 admin admin] PATH_INFO: `/v3/users/c5cb1a83603e437ea576e127c8891fb5` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:43:49.284524 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c4737d-f4c6-4721-9372-f417aba0bb51 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5cb1a83603e437ea576e127c8891fb5)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:43:49.288098 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c4737d-f4c6-4721-9372-f417aba0bb51 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:43:49.294464 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e8bd96-8522-44fc-8e50-53b0d0ff1e69 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:43:49.296076 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e8bd96-8522-44fc-8e50-53b0d0ff1e69 service placement] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:43:49.306500 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1363/2724] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:43:49 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:43:49.307745 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d897d3-7dbc-4279-b204-627f076dc614 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:43:49.308033 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d897d3-7dbc-4279-b204-627f076dc614 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:43:49.308328 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d897d3-7dbc-4279-b204-627f076dc614 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:43:49.334751 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-b5d897d3-7dbc-4279-b204-627f076dc614 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:43:49.335959 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-b5d897d3-7dbc-4279-b204-627f076dc614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:43:49.347773 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-b4c4737d-f4c6-4721-9372-f417aba0bb51 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5cb1a83603e437ea576e127c8891fb5'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:43:49.355730 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1364/2725] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:43:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:43:49.356137 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-b4c4737d-f4c6-4721-9372-f417aba0bb51 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5cb1a83603e437ea576e127c8891fb5'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:43:49.356924 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-b4c4737d-f4c6-4721-9372-f417aba0bb51 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5cb1a83603e437ea576e127c8891fb5'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:43:49.359808 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d897d3-7dbc-4279-b204-627f076dc614 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:43:49.364977 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-b4c4737d-f4c6-4721-9372-f417aba0bb51 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5cb1a83603e437ea576e127c8891fb5'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:43:49.370409 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d897d3-7dbc-4279-b204-627f076dc614 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:43:49.371387 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:43:49.375897 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af378c8b-5705-4ea3-b529-21121a972029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:43:49.376639 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af378c8b-5705-4ea3-b529-21121a972029 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:43:49.376923 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af378c8b-5705-4ea3-b529-21121a972029 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:43:49.377201 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af378c8b-5705-4ea3-b529-21121a972029 admin admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:43:49.377904 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af378c8b-5705-4ea3-b529-21121a972029 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:43:49.378211 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af378c8b-5705-4ea3-b529-21121a972029 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:43:49.379869 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af378c8b-5705-4ea3-b529-21121a972029 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:43:49.380003 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-b4c4737d-f4c6-4721-9372-f417aba0bb51 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5cb1a83603e437ea576e127c8891fb5'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:43:49.387617 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1362/2726] 10.222.0.64 () {58 vars in 1224 bytes} [Tue Aug 2 10:43:49 2022] DELETE /identity/v3/users/c5cb1a83603e437ea576e127c8891fb5 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:43:49.396280 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1365/2727] 10.222.0.64 () {58 vars in 1153 bytes} [Tue Aug 2 10:43:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:43:49.398037 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c4737d-f4c6-4721-9372-f417aba0bb51 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:43:49.399870 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-b4c4737d-f4c6-4721-9372-f417aba0bb51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:43:49.468201 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c4737d-f4c6-4721-9372-f417aba0bb51 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:43:49.473238 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f262857e-2e86-47c2-93fd-8cd33a420cad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:43:49.474235 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f262857e-2e86-47c2-93fd-8cd33a420cad admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:43:49.474478 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f262857e-2e86-47c2-93fd-8cd33a420cad admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:43:49.474887 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f262857e-2e86-47c2-93fd-8cd33a420cad admin admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:43:49.475561 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f262857e-2e86-47c2-93fd-8cd33a420cad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:43:49.478184 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f262857e-2e86-47c2-93fd-8cd33a420cad admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:43:49.509244 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1363/2728] 10.222.0.64 () {60 vars in 1153 bytes} [Tue Aug 2 10:43:49 2022] POST /identity/v3/projects => generated 369 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:43:49.513380 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af378c8b-5705-4ea3-b529-21121a972029 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:43:49.525007 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af378c8b-5705-4ea3-b529-21121a972029 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:43:49.575477 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f262857e-2e86-47c2-93fd-8cd33a420cad admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:43:49.577413 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-f262857e-2e86-47c2-93fd-8cd33a420cad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:43:49.593511 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f3d998-cbff-422a-97bc-57344521d43b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:43:49.594142 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f3d998-cbff-422a-97bc-57344521d43b admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:43:49.594434 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f3d998-cbff-422a-97bc-57344521d43b admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:43:49.594716 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f3d998-cbff-422a-97bc-57344521d43b admin admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:43:49.595405 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f3d998-cbff-422a-97bc-57344521d43b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:43:49.597392 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f3d998-cbff-422a-97bc-57344521d43b admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:43:49.610910 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:43:49.630653 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1366/2729] 10.222.0.64 () {60 vars in 1144 bytes} [Tue Aug 2 10:43:49 2022] POST /identity/v3/users => generated 285 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:43:49.631760 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f3d998-cbff-422a-97bc-57344521d43b admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:43:49.645926 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f3d998-cbff-422a-97bc-57344521d43b admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:43:49.652880 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab828b3-1ef0-41f4-a458-b11733f0c40e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:43:49.653512 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab828b3-1ef0-41f4-a458-b11733f0c40e admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:43:49.653855 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab828b3-1ef0-41f4-a458-b11733f0c40e admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:43:49.654076 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab828b3-1ef0-41f4-a458-b11733f0c40e admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:43:49.655086 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab828b3-1ef0-41f4-a458-b11733f0c40e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:43:49.656420 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab828b3-1ef0-41f4-a458-b11733f0c40e admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:43:49.661570 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f262857e-2e86-47c2-93fd-8cd33a420cad admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:43:49.668289 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1367/2730] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:43:49 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:43:49.673421 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab828b3-1ef0-41f4-a458-b11733f0c40e admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:43:49.677718 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1048e679-db45-4745-9e79-51b9122452f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:43:49.678384 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1048e679-db45-4745-9e79-51b9122452f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:43:49.678668 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1048e679-db45-4745-9e79-51b9122452f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:43:49.678945 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1048e679-db45-4745-9e79-51b9122452f5 admin admin] PATH_INFO: `/v3/projects/25fd81d1b89c4a5788588d147622baed` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:43:49.680178 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1048e679-db45-4745-9e79-51b9122452f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=25fd81d1b89c4a5788588d147622baed)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:43:49.682740 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1048e679-db45-4745-9e79-51b9122452f5 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:43:49.685040 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab828b3-1ef0-41f4-a458-b11733f0c40e admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:43:49.689722 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b680531-3583-4130-88fd-4be90110c8fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:43:49.690330 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b680531-3583-4130-88fd-4be90110c8fd admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:43:49.690616 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b680531-3583-4130-88fd-4be90110c8fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:43:49.690890 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b680531-3583-4130-88fd-4be90110c8fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:43:49.691686 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b680531-3583-4130-88fd-4be90110c8fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:43:49.693225 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b680531-3583-4130-88fd-4be90110c8fd admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:43:49.704577 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1368/2731] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:43:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:43:49.708589 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b680531-3583-4130-88fd-4be90110c8fd admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:43:49.720413 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b680531-3583-4130-88fd-4be90110c8fd admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:43:49.725688 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9eb1f60-c1ec-4e98-902e-ae748842f863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:43:49.726365 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eb1f60-c1ec-4e98-902e-ae748842f863 admin admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:43:49.726650 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eb1f60-c1ec-4e98-902e-ae748842f863 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:43:49.727422 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eb1f60-c1ec-4e98-902e-ae748842f863 admin admin] PATH_INFO: `/v3/projects/ff47e7a203a04076b88a870e53099649/users/91cb79b35fd9427fbdab6ce9f9be9ed3/roles/66c1074780954f8e87ed11a5c4e8c36a` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:43:49.743342 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-1048e679-db45-4745-9e79-51b9122452f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25fd81d1b89c4a5788588d147622baed', 'request_id': 'req-1048e679-db45-4745-9e79-51b9122452f5'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:43:49.743657 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9eb1f60-c1ec-4e98-902e-ae748842f863 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff47e7a203a04076b88a870e53099649, user_id=91cb79b35fd9427fbdab6ce9f9be9ed3, role_id=66c1074780954f8e87ed11a5c4e8c36a)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:43:49.746392 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9eb1f60-c1ec-4e98-902e-ae748842f863 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:43:49.748948 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-1048e679-db45-4745-9e79-51b9122452f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25fd81d1b89c4a5788588d147622baed', 'request_id': 'req-1048e679-db45-4745-9e79-51b9122452f5'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:43:49.756665 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-1048e679-db45-4745-9e79-51b9122452f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25fd81d1b89c4a5788588d147622baed', 'request_id': 'req-1048e679-db45-4745-9e79-51b9122452f5'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:43:49.760719 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1369/2732] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 10:43:49 2022] PUT /identity/v3/projects/ff47e7a203a04076b88a870e53099649/users/91cb79b35fd9427fbdab6ce9f9be9ed3/roles/66c1074780954f8e87ed11a5c4e8c36a => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:43:49.772905 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183f2e95-b342-4136-8104-862f3f8bb91a None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:43:49.773194 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183f2e95-b342-4136-8104-862f3f8bb91a None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:43:49.773492 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183f2e95-b342-4136-8104-862f3f8bb91a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:43:49.779576 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-1048e679-db45-4745-9e79-51b9122452f5 admin admin] The token cache is being invalidate because project 25fd81d1b89c4a5788588d147622baed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80154) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:43:49.780398 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-1048e679-db45-4745-9e79-51b9122452f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:43:49.781104 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-1048e679-db45-4745-9e79-51b9122452f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:43:49.782656 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1364/2733] 10.222.0.64 () {58 vars in 1233 bytes} [Tue Aug 2 10:43:49 2022] DELETE /identity/v3/projects/25fd81d1b89c4a5788588d147622baed => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:43:49.788510 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4bd699-761c-41a1-9305-b931cfb73201 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:43:49.788820 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4bd699-761c-41a1-9305-b931cfb73201 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:43:49.789087 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4bd699-761c-41a1-9305-b931cfb73201 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:43:49.796877 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=90,keystone:DELETE=8,keystone:INSERT=2 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:43:49.811278 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-5d4bd699-761c-41a1-9305-b931cfb73201 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:43:49.812530 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-5d4bd699-761c-41a1-9305-b931cfb73201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:43:49.815260 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-183f2e95-b342-4136-8104-862f3f8bb91a None None] MFA Rules not processed for user `91cb79b35fd9427fbdab6ce9f9be9ed3`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:43:49.816482 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-183f2e95-b342-4136-8104-862f3f8bb91a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:43:49.876027 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1365/2734] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:43:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:43:49.882800 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1370/2735] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:43:49 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:43:49.884948 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2eff4e-1cb8-41d4-aed7-d4fed725680e None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:43:49.885302 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2eff4e-1cb8-41d4-aed7-d4fed725680e None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:43:49.885642 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2eff4e-1cb8-41d4-aed7-d4fed725680e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:43:49.886965 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183f2e95-b342-4136-8104-862f3f8bb91a None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:43:49.888404 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-183f2e95-b342-4136-8104-862f3f8bb91a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:43:49.902519 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183f2e95-b342-4136-8104-862f3f8bb91a None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:43:49.905966 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-db2eff4e-1cb8-41d4-aed7-d4fed725680e None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:43:49.907068 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dec6fbb-185f-4f2e-9e82-870e292a235d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:43:49.907227 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-db2eff4e-1cb8-41d4-aed7-d4fed725680e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:43:49.907689 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dec6fbb-185f-4f2e-9e82-870e292a235d admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:43:49.908003 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dec6fbb-185f-4f2e-9e82-870e292a235d admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:43:49.908259 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dec6fbb-185f-4f2e-9e82-870e292a235d admin admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:43:49.908943 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dec6fbb-185f-4f2e-9e82-870e292a235d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:43:49.910842 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dec6fbb-185f-4f2e-9e82-870e292a235d admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:43:49.915669 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1366/2736] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:43:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:43:49.919762 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2eff4e-1cb8-41d4-aed7-d4fed725680e None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:43:49.929992 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2eff4e-1cb8-41d4-aed7-d4fed725680e None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:43:49.936221 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7849afd0-6983-4695-a3fd-5095b9df85f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:43:49.936866 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7849afd0-6983-4695-a3fd-5095b9df85f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:43:49.937154 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7849afd0-6983-4695-a3fd-5095b9df85f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:43:49.937433 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7849afd0-6983-4695-a3fd-5095b9df85f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:43:49.938141 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7849afd0-6983-4695-a3fd-5095b9df85f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:43:49.938451 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7849afd0-6983-4695-a3fd-5095b9df85f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:43:49.940048 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7849afd0-6983-4695-a3fd-5095b9df85f1 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:43:49.944299 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1371/2737] 10.222.0.64 () {60 vars in 1153 bytes} [Tue Aug 2 10:43:49 2022] POST /identity/v3/projects => generated 367 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:43:49.945624 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dec6fbb-185f-4f2e-9e82-870e292a235d admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:43:49.955481 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1367/2738] 10.222.0.64 () {58 vars in 1153 bytes} [Tue Aug 2 10:43:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:43:49.956449 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7849afd0-6983-4695-a3fd-5095b9df85f1 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:43:49.967199 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7849afd0-6983-4695-a3fd-5095b9df85f1 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:43:50.009899 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dec6fbb-185f-4f2e-9e82-870e292a235d admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:43:50.014894 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd169cf-59d2-43f9-aaa3-88d38bd6ac11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:43:50.015515 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd169cf-59d2-43f9-aaa3-88d38bd6ac11 admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:43:50.015806 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd169cf-59d2-43f9-aaa3-88d38bd6ac11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:43:50.016087 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd169cf-59d2-43f9-aaa3-88d38bd6ac11 admin admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:43:50.016800 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd169cf-59d2-43f9-aaa3-88d38bd6ac11 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:43:50.018747 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd169cf-59d2-43f9-aaa3-88d38bd6ac11 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:43:50.025996 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930daaf7-0495-45ca-a36a-f0b2fe849f9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:43:50.026613 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930daaf7-0495-45ca-a36a-f0b2fe849f9a admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:43:50.026899 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930daaf7-0495-45ca-a36a-f0b2fe849f9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:43:50.027179 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930daaf7-0495-45ca-a36a-f0b2fe849f9a admin admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:43:50.027844 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930daaf7-0495-45ca-a36a-f0b2fe849f9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:43:50.029778 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930daaf7-0495-45ca-a36a-f0b2fe849f9a admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:43:50.034836 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:43:50.044782 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1372/2739] 10.222.0.64 () {60 vars in 1144 bytes} [Tue Aug 2 10:43:49 2022] POST /identity/v3/users => generated 290 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:43:50.046509 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd169cf-59d2-43f9-aaa3-88d38bd6ac11 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:43:50.062525 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1368/2740] 10.222.0.64 () {60 vars in 1153 bytes} [Tue Aug 2 10:43:49 2022] POST /identity/v3/projects => generated 373 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:43:50.063054 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd169cf-59d2-43f9-aaa3-88d38bd6ac11 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:43:50.063753 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930daaf7-0495-45ca-a36a-f0b2fe849f9a admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:43:50.121904 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d793269-2aab-4c9c-8f33-50c1f9bb60ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:43:50.122578 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d793269-2aab-4c9c-8f33-50c1f9bb60ec admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:43:50.122866 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d793269-2aab-4c9c-8f33-50c1f9bb60ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:43:50.123172 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d793269-2aab-4c9c-8f33-50c1f9bb60ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:43:50.123878 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d793269-2aab-4c9c-8f33-50c1f9bb60ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:43:50.125601 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930daaf7-0495-45ca-a36a-f0b2fe849f9a admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:43:50.125772 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d793269-2aab-4c9c-8f33-50c1f9bb60ec admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:43:50.130330 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18491c8e-2102-43fc-a291-cab8cfeda2b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:43:50.130945 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18491c8e-2102-43fc-a291-cab8cfeda2b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:43:50.131231 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18491c8e-2102-43fc-a291-cab8cfeda2b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:43:50.131510 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18491c8e-2102-43fc-a291-cab8cfeda2b3 admin admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:43:50.132177 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18491c8e-2102-43fc-a291-cab8cfeda2b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:43:50.134169 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18491c8e-2102-43fc-a291-cab8cfeda2b3 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:43:50.135840 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1373/2741] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:43:50 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:43:50.139994 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d793269-2aab-4c9c-8f33-50c1f9bb60ec admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:43:50.150582 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d793269-2aab-4c9c-8f33-50c1f9bb60ec admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:43:50.151949 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:43:50.156399 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f9c9a7-4402-498e-ac06-5705a7601dc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:43:50.156966 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f9c9a7-4402-498e-ac06-5705a7601dc8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:43:50.157305 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f9c9a7-4402-498e-ac06-5705a7601dc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:43:50.157584 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f9c9a7-4402-498e-ac06-5705a7601dc8 admin admin] PATH_INFO: `/v3/projects/a6ac771501c048bf8c96bc1d7a107ecb/users/1f26dc076beb4cd396b8728ca9d47443/roles/1000cf318fa14915b8091599eaead5de` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:43:50.158958 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1369/2742] 10.222.0.64 () {60 vars in 1144 bytes} [Tue Aug 2 10:43:50 2022] POST /identity/v3/users => generated 287 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:43:50.162663 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18491c8e-2102-43fc-a291-cab8cfeda2b3 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:43:50.173340 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18491c8e-2102-43fc-a291-cab8cfeda2b3 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:43:50.173730 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f9c9a7-4402-498e-ac06-5705a7601dc8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6ac771501c048bf8c96bc1d7a107ecb, user_id=1f26dc076beb4cd396b8728ca9d47443, role_id=1000cf318fa14915b8091599eaead5de)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:43:50.176417 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f9c9a7-4402-498e-ac06-5705a7601dc8 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:43:50.178413 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0033a00-f779-4ab7-a53c-501cc5b8f59f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:43:50.179042 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0033a00-f779-4ab7-a53c-501cc5b8f59f admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:43:50.179324 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0033a00-f779-4ab7-a53c-501cc5b8f59f admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:43:50.179597 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0033a00-f779-4ab7-a53c-501cc5b8f59f admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:43:50.180414 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0033a00-f779-4ab7-a53c-501cc5b8f59f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:43:50.181992 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0033a00-f779-4ab7-a53c-501cc5b8f59f admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:43:50.190607 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1374/2743] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 10:43:50 2022] PUT /identity/v3/projects/a6ac771501c048bf8c96bc1d7a107ecb/users/1f26dc076beb4cd396b8728ca9d47443/roles/1000cf318fa14915b8091599eaead5de => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:43:50.192305 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1370/2744] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:43:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:43:50.194690 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f9c9a7-4402-498e-ac06-5705a7601dc8 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:43:50.196235 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0033a00-f779-4ab7-a53c-501cc5b8f59f admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:43:50.206555 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0033a00-f779-4ab7-a53c-501cc5b8f59f admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:43:50.256007 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f9c9a7-4402-498e-ac06-5705a7601dc8 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:43:50.261207 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d348aab8-8e6d-4e7e-b803-ca223e48ea71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:43:50.261900 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d348aab8-8e6d-4e7e-b803-ca223e48ea71 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:43:50.262222 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d348aab8-8e6d-4e7e-b803-ca223e48ea71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:43:50.262511 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d348aab8-8e6d-4e7e-b803-ca223e48ea71 admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:43:50.263258 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d348aab8-8e6d-4e7e-b803-ca223e48ea71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:43:50.263807 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514089b9-56ed-4839-84f0-e31887f4a04a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:43:50.264469 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514089b9-56ed-4839-84f0-e31887f4a04a admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:43:50.264755 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514089b9-56ed-4839-84f0-e31887f4a04a admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:43:50.265012 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d348aab8-8e6d-4e7e-b803-ca223e48ea71 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:43:50.265137 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514089b9-56ed-4839-84f0-e31887f4a04a admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:43:50.265842 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514089b9-56ed-4839-84f0-e31887f4a04a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:43:50.267458 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514089b9-56ed-4839-84f0-e31887f4a04a admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:43:50.276136 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1375/2745] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:43:50 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:43:50.280177 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d348aab8-8e6d-4e7e-b803-ca223e48ea71 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:43:50.281469 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1371/2746] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:43:50 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:43:50.282366 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514089b9-56ed-4839-84f0-e31887f4a04a admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:43:50.290882 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d348aab8-8e6d-4e7e-b803-ca223e48ea71 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:43:50.292812 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514089b9-56ed-4839-84f0-e31887f4a04a admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:43:50.295566 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf2a68a-7d9d-4aa9-b2b3-833d9eaf2d7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:43:50.296300 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf2a68a-7d9d-4aa9-b2b3-833d9eaf2d7a admin admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:43:50.296560 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf2a68a-7d9d-4aa9-b2b3-833d9eaf2d7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:43:50.296838 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf2a68a-7d9d-4aa9-b2b3-833d9eaf2d7a admin admin] PATH_INFO: `/v3/projects/a6ac771501c048bf8c96bc1d7a107ecb/users/1f26dc076beb4cd396b8728ca9d47443/roles/1000cf318fa14915b8091599eaead5de` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:43:50.297499 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89af783b-6f18-4969-8533-db71153df1bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:43:50.298084 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89af783b-6f18-4969-8533-db71153df1bb admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:43:50.298369 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89af783b-6f18-4969-8533-db71153df1bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:43:50.298642 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89af783b-6f18-4969-8533-db71153df1bb admin admin] PATH_INFO: `/v3/projects/91f790e57c874588b0e46e22d00afe3e/users/672442a55e9b47ccb8da313f8d3f4428/roles/66c1074780954f8e87ed11a5c4e8c36a` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:43:50.298881 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf2a68a-7d9d-4aa9-b2b3-833d9eaf2d7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6ac771501c048bf8c96bc1d7a107ecb, user_id=1f26dc076beb4cd396b8728ca9d47443, role_id=1000cf318fa14915b8091599eaead5de)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:43:50.301556 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf2a68a-7d9d-4aa9-b2b3-833d9eaf2d7a admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:43:50.312961 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1376/2747] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 10:43:50 2022] PUT /identity/v3/projects/a6ac771501c048bf8c96bc1d7a107ecb/users/1f26dc076beb4cd396b8728ca9d47443/roles/1000cf318fa14915b8091599eaead5de => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:43:50.313609 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89af783b-6f18-4969-8533-db71153df1bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=91f790e57c874588b0e46e22d00afe3e, user_id=672442a55e9b47ccb8da313f8d3f4428, role_id=66c1074780954f8e87ed11a5c4e8c36a)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:43:50.316229 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89af783b-6f18-4969-8533-db71153df1bb admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:43:50.323824 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ddd58b-f0a6-4a37-8998-395993996765 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:43:50.324270 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ddd58b-f0a6-4a37-8998-395993996765 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:43:50.324664 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ddd58b-f0a6-4a37-8998-395993996765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:43:50.330351 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1372/2748] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 10:43:50 2022] PUT /identity/v3/projects/91f790e57c874588b0e46e22d00afe3e/users/672442a55e9b47ccb8da313f8d3f4428/roles/66c1074780954f8e87ed11a5c4e8c36a => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:43:50.335353 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cd8674-fb64-495b-88cf-1ec0528a1f32 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:43:50.335653 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cd8674-fb64-495b-88cf-1ec0528a1f32 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:43:50.335963 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cd8674-fb64-495b-88cf-1ec0528a1f32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:43:50.355114 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-20ddd58b-f0a6-4a37-8998-395993996765 None None] MFA Rules not processed for user `1f26dc076beb4cd396b8728ca9d47443`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:43:50.355610 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-08cd8674-fb64-495b-88cf-1ec0528a1f32 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:43:50.356362 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-20ddd58b-f0a6-4a37-8998-395993996765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:43:50.356483 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-08cd8674-fb64-495b-88cf-1ec0528a1f32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:43:50.407820 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:43:50.423436 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1373/2749] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:43:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:43:50.434248 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cd8674-fb64-495b-88cf-1ec0528a1f32 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:43:50.435624 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-08cd8674-fb64-495b-88cf-1ec0528a1f32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:43:50.437257 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1377/2750] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:43:50 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:43:50.448448 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ddd58b-f0a6-4a37-8998-395993996765 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:43:50.449802 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-20ddd58b-f0a6-4a37-8998-395993996765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:43:50.493361 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cd8674-fb64-495b-88cf-1ec0528a1f32 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:43:50.497040 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603c096b-0574-4435-a963-7d866fc19288 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:43:50.497657 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603c096b-0574-4435-a963-7d866fc19288 service neutron] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:43:50.497940 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603c096b-0574-4435-a963-7d866fc19288 service neutron] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:43:50.498216 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603c096b-0574-4435-a963-7d866fc19288 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:43:50.508031 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603c096b-0574-4435-a963-7d866fc19288 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:43:50.510062 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603c096b-0574-4435-a963-7d866fc19288 service neutron] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:43:50.519710 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1374/2751] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:43:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:43:50.520577 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ddd58b-f0a6-4a37-8998-395993996765 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:43:50.524910 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e95cc5e-cbaf-439e-adb8-a19c0e3e9afe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d7de62b9ee4039bdc79627ac7fb499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:43:50.525616 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e95cc5e-cbaf-439e-adb8-a19c0e3e9afe service cinder] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:43:50.525804 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e95cc5e-cbaf-439e-adb8-a19c0e3e9afe service cinder] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:43:50.526080 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e95cc5e-cbaf-439e-adb8-a19c0e3e9afe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:43:50.535477 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e95cc5e-cbaf-439e-adb8-a19c0e3e9afe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:43:50.536977 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e95cc5e-cbaf-439e-adb8-a19c0e3e9afe service cinder] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:43:50.548063 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1378/2752] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:43:50 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:43:52.350213 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603c096b-0574-4435-a963-7d866fc19288 service neutron] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:43:52.355860 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:43:52.371434 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603c096b-0574-4435-a963-7d866fc19288 service neutron] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:43:52.385593 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca6fcfa-2874-45c4-bf86-0f67b64eaa84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:43:52.387042 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca6fcfa-2874-45c4-bf86-0f67b64eaa84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:43:52.387664 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca6fcfa-2874-45c4-bf86-0f67b64eaa84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:43:52.388212 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca6fcfa-2874-45c4-bf86-0f67b64eaa84 admin admin] PATH_INFO: `/v3/users/91cb79b35fd9427fbdab6ce9f9be9ed3` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:43:52.390398 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca6fcfa-2874-45c4-bf86-0f67b64eaa84 admin admin] RBAC: Authorizing `identity:delete_user(user_id=91cb79b35fd9427fbdab6ce9f9be9ed3)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:43:52.392730 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca6fcfa-2874-45c4-bf86-0f67b64eaa84 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:43:52.459504 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-bca6fcfa-2874-45c4-bf86-0f67b64eaa84 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91cb79b35fd9427fbdab6ce9f9be9ed3'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:43:52.473236 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-bca6fcfa-2874-45c4-bf86-0f67b64eaa84 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91cb79b35fd9427fbdab6ce9f9be9ed3'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:43:52.474174 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-bca6fcfa-2874-45c4-bf86-0f67b64eaa84 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91cb79b35fd9427fbdab6ce9f9be9ed3'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:43:52.484745 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-bca6fcfa-2874-45c4-bf86-0f67b64eaa84 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91cb79b35fd9427fbdab6ce9f9be9ed3'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:43:52.507175 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-bca6fcfa-2874-45c4-bf86-0f67b64eaa84 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91cb79b35fd9427fbdab6ce9f9be9ed3'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:43:52.521549 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1375/2753] 10.222.0.64 () {58 vars in 1224 bytes} [Tue Aug 2 10:43:52 2022] DELETE /identity/v3/users/91cb79b35fd9427fbdab6ce9f9be9ed3 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:43:52.523363 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a473d2-b7ac-4d8d-a62e-7fe21fd1339b None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:43:52.524035 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a473d2-b7ac-4d8d-a62e-7fe21fd1339b None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:43:52.524035 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a473d2-b7ac-4d8d-a62e-7fe21fd1339b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:43:52.546098 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-95a473d2-b7ac-4d8d-a62e-7fe21fd1339b None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:43:52.547365 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-95a473d2-b7ac-4d8d-a62e-7fe21fd1339b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:43:52.650861 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1379/2754] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:43:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:43:52.664070 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca6fcfa-2874-45c4-bf86-0f67b64eaa84 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:43:52.666378 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-bca6fcfa-2874-45c4-bf86-0f67b64eaa84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:43:52.744222 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca6fcfa-2874-45c4-bf86-0f67b64eaa84 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:43:52.750441 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c725c179-ce8d-4522-ae28-8f7a587659a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:43:52.750441 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c725c179-ce8d-4522-ae28-8f7a587659a6 service neutron] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:43:52.750441 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c725c179-ce8d-4522-ae28-8f7a587659a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:43:52.750441 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c725c179-ce8d-4522-ae28-8f7a587659a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:43:52.763692 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c725c179-ce8d-4522-ae28-8f7a587659a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:43:52.764913 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c725c179-ce8d-4522-ae28-8f7a587659a6 service neutron] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:43:52.780533 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1376/2755] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:43:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:43:52.805816 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a473d2-b7ac-4d8d-a62e-7fe21fd1339b None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:43:52.816048 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a473d2-b7ac-4d8d-a62e-7fe21fd1339b None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:43:52.822725 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b57470-5b38-4efc-b81b-bbdb887a6ae5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:43:52.829489 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b57470-5b38-4efc-b81b-bbdb887a6ae5 service neutron] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:43:52.829937 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b57470-5b38-4efc-b81b-bbdb887a6ae5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:43:52.830226 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b57470-5b38-4efc-b81b-bbdb887a6ae5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:43:52.834169 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-23b57470-5b38-4efc-b81b-bbdb887a6ae5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:43:52.859845 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b57470-5b38-4efc-b81b-bbdb887a6ae5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:43:52.861405 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b57470-5b38-4efc-b81b-bbdb887a6ae5 service neutron] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:43:52.876689 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1380/2756] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:43:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:43:53.704339 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c725c179-ce8d-4522-ae28-8f7a587659a6 service neutron] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:43:53.706197 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-c725c179-ce8d-4522-ae28-8f7a587659a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:43:53.734630 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c725c179-ce8d-4522-ae28-8f7a587659a6 service neutron] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:43:53.752604 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939f407e-52f1-425e-ad62-1a83a543fe67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:43:53.754844 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939f407e-52f1-425e-ad62-1a83a543fe67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:43:53.755410 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939f407e-52f1-425e-ad62-1a83a543fe67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:43:53.755894 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939f407e-52f1-425e-ad62-1a83a543fe67 admin admin] PATH_INFO: `/v3/projects/ff47e7a203a04076b88a870e53099649` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:43:53.757709 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939f407e-52f1-425e-ad62-1a83a543fe67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff47e7a203a04076b88a870e53099649)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:43:53.763146 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939f407e-52f1-425e-ad62-1a83a543fe67 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:43:53.839895 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-939f407e-52f1-425e-ad62-1a83a543fe67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff47e7a203a04076b88a870e53099649', 'request_id': 'req-939f407e-52f1-425e-ad62-1a83a543fe67'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:43:53.846951 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-939f407e-52f1-425e-ad62-1a83a543fe67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff47e7a203a04076b88a870e53099649', 'request_id': 'req-939f407e-52f1-425e-ad62-1a83a543fe67'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:43:53.854232 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-939f407e-52f1-425e-ad62-1a83a543fe67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff47e7a203a04076b88a870e53099649', 'request_id': 'req-939f407e-52f1-425e-ad62-1a83a543fe67'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:43:53.864841 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:43:53.870904 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-939f407e-52f1-425e-ad62-1a83a543fe67 admin admin] The token cache is being invalidate because project ff47e7a203a04076b88a870e53099649 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80154) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:43:53.872078 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-939f407e-52f1-425e-ad62-1a83a543fe67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:43:53.873504 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-939f407e-52f1-425e-ad62-1a83a543fe67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:43:53.875976 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1377/2757] 10.222.0.64 () {58 vars in 1233 bytes} [Tue Aug 2 10:43:53 2022] DELETE /identity/v3/projects/ff47e7a203a04076b88a870e53099649 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:43:53.880664 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b57470-5b38-4efc-b81b-bbdb887a6ae5 service neutron] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:43:53.892607 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b57470-5b38-4efc-b81b-bbdb887a6ae5 service neutron] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:43:53.991317 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:43:54.000204 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e86c6ee-0251-4b1c-9f00-7f5d00deb345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:43:54.001010 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e86c6ee-0251-4b1c-9f00-7f5d00deb345 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:43:54.001343 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e86c6ee-0251-4b1c-9f00-7f5d00deb345 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:43:54.001621 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e86c6ee-0251-4b1c-9f00-7f5d00deb345 admin admin] PATH_INFO: `/v3/users/1f26dc076beb4cd396b8728ca9d47443` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:43:54.003034 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e86c6ee-0251-4b1c-9f00-7f5d00deb345 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f26dc076beb4cd396b8728ca9d47443)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:43:54.005906 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e86c6ee-0251-4b1c-9f00-7f5d00deb345 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:43:54.110547 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-5e86c6ee-0251-4b1c-9f00-7f5d00deb345 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f26dc076beb4cd396b8728ca9d47443'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:43:54.123140 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-5e86c6ee-0251-4b1c-9f00-7f5d00deb345 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f26dc076beb4cd396b8728ca9d47443'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:43:54.136897 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-5e86c6ee-0251-4b1c-9f00-7f5d00deb345 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f26dc076beb4cd396b8728ca9d47443'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:43:54.137596 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-5e86c6ee-0251-4b1c-9f00-7f5d00deb345 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f26dc076beb4cd396b8728ca9d47443'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:43:54.138336 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-5e86c6ee-0251-4b1c-9f00-7f5d00deb345 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f26dc076beb4cd396b8728ca9d47443'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:43:54.186921 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1381/2758] 10.222.0.64 () {58 vars in 1224 bytes} [Tue Aug 2 10:43:53 2022] DELETE /identity/v3/users/1f26dc076beb4cd396b8728ca9d47443 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:43:54.961616 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939f407e-52f1-425e-ad62-1a83a543fe67 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:43:54.963605 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-939f407e-52f1-425e-ad62-1a83a543fe67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:43:54.993672 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939f407e-52f1-425e-ad62-1a83a543fe67 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:43:55.120740 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c66d768-ad7f-411f-8c32-16d06f501f0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:43:55.121680 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c66d768-ad7f-411f-8c32-16d06f501f0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:43:55.122066 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c66d768-ad7f-411f-8c32-16d06f501f0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:43:55.122448 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c66d768-ad7f-411f-8c32-16d06f501f0c admin admin] PATH_INFO: `/v3/projects/a6ac771501c048bf8c96bc1d7a107ecb` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:43:55.123958 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c66d768-ad7f-411f-8c32-16d06f501f0c admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6ac771501c048bf8c96bc1d7a107ecb)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:43:55.126501 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c66d768-ad7f-411f-8c32-16d06f501f0c admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:43:55.199420 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-6c66d768-ad7f-411f-8c32-16d06f501f0c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6ac771501c048bf8c96bc1d7a107ecb', 'request_id': 'req-6c66d768-ad7f-411f-8c32-16d06f501f0c'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:43:55.207244 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-6c66d768-ad7f-411f-8c32-16d06f501f0c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6ac771501c048bf8c96bc1d7a107ecb', 'request_id': 'req-6c66d768-ad7f-411f-8c32-16d06f501f0c'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:43:55.216225 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-6c66d768-ad7f-411f-8c32-16d06f501f0c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6ac771501c048bf8c96bc1d7a107ecb', 'request_id': 'req-6c66d768-ad7f-411f-8c32-16d06f501f0c'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:43:55.235279 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-6c66d768-ad7f-411f-8c32-16d06f501f0c admin admin] The token cache is being invalidate because project a6ac771501c048bf8c96bc1d7a107ecb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80154) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:43:55.235764 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-6c66d768-ad7f-411f-8c32-16d06f501f0c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:43:55.236755 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-6c66d768-ad7f-411f-8c32-16d06f501f0c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:43:55.238975 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1378/2759] 10.222.0.64 () {58 vars in 1233 bytes} [Tue Aug 2 10:43:54 2022] DELETE /identity/v3/projects/a6ac771501c048bf8c96bc1d7a107ecb => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:43:55.245304 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40b6950-75fc-4aa1-b43e-05d78af0b136 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:43:55.245976 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40b6950-75fc-4aa1-b43e-05d78af0b136 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:43:55.245976 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40b6950-75fc-4aa1-b43e-05d78af0b136 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:43:55.282881 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-f40b6950-75fc-4aa1-b43e-05d78af0b136 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:43:55.287792 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-f40b6950-75fc-4aa1-b43e-05d78af0b136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:43:55.404448 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1382/2760] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:43:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:43:55.412736 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30079ea7-28ef-4978-81a4-509b785a83cc None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:43:55.413400 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30079ea7-28ef-4978-81a4-509b785a83cc None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:43:55.413782 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30079ea7-28ef-4978-81a4-509b785a83cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:43:55.435996 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-30079ea7-28ef-4978-81a4-509b785a83cc None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:43:55.438305 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-30079ea7-28ef-4978-81a4-509b785a83cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:43:55.456573 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1379/2761] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:43:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:43:55.457260 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40b6950-75fc-4aa1-b43e-05d78af0b136 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:43:55.470289 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40b6950-75fc-4aa1-b43e-05d78af0b136 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:43:55.476479 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bacf9b-a17f-4abe-88e7-1ad772a06568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:43:55.477486 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bacf9b-a17f-4abe-88e7-1ad772a06568 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:43:55.477976 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bacf9b-a17f-4abe-88e7-1ad772a06568 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:43:55.478440 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bacf9b-a17f-4abe-88e7-1ad772a06568 admin admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:43:55.479399 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bacf9b-a17f-4abe-88e7-1ad772a06568 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:43:55.479965 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bacf9b-a17f-4abe-88e7-1ad772a06568 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:43:55.482247 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bacf9b-a17f-4abe-88e7-1ad772a06568 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:43:55.497872 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1383/2762] 10.222.0.64 () {58 vars in 1153 bytes} [Tue Aug 2 10:43:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:43:55.502241 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30079ea7-28ef-4978-81a4-509b785a83cc None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:43:55.524083 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30079ea7-28ef-4978-81a4-509b785a83cc None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:43:55.534135 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941c2be6-d793-4877-a33e-b88dee6e8b50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:43:55.534777 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941c2be6-d793-4877-a33e-b88dee6e8b50 admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:43:55.535060 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941c2be6-d793-4877-a33e-b88dee6e8b50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:43:55.535331 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941c2be6-d793-4877-a33e-b88dee6e8b50 admin admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:43:55.536016 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941c2be6-d793-4877-a33e-b88dee6e8b50 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:43:55.537994 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941c2be6-d793-4877-a33e-b88dee6e8b50 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:43:55.601211 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1380/2763] 10.222.0.64 () {60 vars in 1153 bytes} [Tue Aug 2 10:43:55 2022] POST /identity/v3/projects => generated 379 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:43:55.605182 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bacf9b-a17f-4abe-88e7-1ad772a06568 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:43:55.617194 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bacf9b-a17f-4abe-88e7-1ad772a06568 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:43:55.694322 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f006e27c-c185-4f50-b43f-8a48a4f24a57 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:43:55.694732 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f006e27c-c185-4f50-b43f-8a48a4f24a57 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:43:55.695872 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f006e27c-c185-4f50-b43f-8a48a4f24a57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:43:55.719632 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddf83e0-1773-4dfd-99a6-1a403974499b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:43:55.720410 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddf83e0-1773-4dfd-99a6-1a403974499b admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:43:55.720825 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddf83e0-1773-4dfd-99a6-1a403974499b admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:43:55.721157 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddf83e0-1773-4dfd-99a6-1a403974499b admin admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:43:55.721841 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddf83e0-1773-4dfd-99a6-1a403974499b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:43:55.724447 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddf83e0-1773-4dfd-99a6-1a403974499b admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:43:55.729251 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-f006e27c-c185-4f50-b43f-8a48a4f24a57 None None] MFA Rules not processed for user `672442a55e9b47ccb8da313f8d3f4428`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:43:55.730593 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-f006e27c-c185-4f50-b43f-8a48a4f24a57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:43:55.748447 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:43:55.772326 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1384/2764] 10.222.0.64 () {60 vars in 1144 bytes} [Tue Aug 2 10:43:55 2022] POST /identity/v3/users => generated 290 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:43:55.773552 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddf83e0-1773-4dfd-99a6-1a403974499b admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:43:55.789099 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddf83e0-1773-4dfd-99a6-1a403974499b admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:43:55.801990 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:43:55.807978 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bb8c82-fff5-42f3-99cc-77fa4c4b5a51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:43:55.809131 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bb8c82-fff5-42f3-99cc-77fa4c4b5a51 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:43:55.809419 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bb8c82-fff5-42f3-99cc-77fa4c4b5a51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:43:55.809773 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bb8c82-fff5-42f3-99cc-77fa4c4b5a51 admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:43:55.810861 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bb8c82-fff5-42f3-99cc-77fa4c4b5a51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:43:55.814375 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bb8c82-fff5-42f3-99cc-77fa4c4b5a51 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:43:55.832362 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1381/2765] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:43:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:43:55.833364 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f006e27c-c185-4f50-b43f-8a48a4f24a57 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:43:55.833594 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1385/2766] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:43:55 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:43:55.834993 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-f006e27c-c185-4f50-b43f-8a48a4f24a57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:43:55.837361 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bb8c82-fff5-42f3-99cc-77fa4c4b5a51 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:43:55.851360 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f006e27c-c185-4f50-b43f-8a48a4f24a57 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:43:55.852849 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bb8c82-fff5-42f3-99cc-77fa4c4b5a51 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:43:55.858264 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df98d75-e8ea-40e6-ac93-d1956387a046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:43:55.859011 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df98d75-e8ea-40e6-ac93-d1956387a046 admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:43:55.859295 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df98d75-e8ea-40e6-ac93-d1956387a046 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:43:55.859489 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b3cc36-1682-4bfe-aa27-08d0d07da541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:43:55.859620 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df98d75-e8ea-40e6-ac93-d1956387a046 admin admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:43:55.862312 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df98d75-e8ea-40e6-ac93-d1956387a046 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:43:55.862594 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b3cc36-1682-4bfe-aa27-08d0d07da541 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:43:55.862594 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b3cc36-1682-4bfe-aa27-08d0d07da541 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:43:55.862594 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b3cc36-1682-4bfe-aa27-08d0d07da541 admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:43:55.862594 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b3cc36-1682-4bfe-aa27-08d0d07da541 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:43:55.862989 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df98d75-e8ea-40e6-ac93-d1956387a046 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:43:55.864940 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b3cc36-1682-4bfe-aa27-08d0d07da541 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:43:55.908542 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1386/2767] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:43:55 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:43:55.912329 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b3cc36-1682-4bfe-aa27-08d0d07da541 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:43:55.925842 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b3cc36-1682-4bfe-aa27-08d0d07da541 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:43:55.940860 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9e376c-0bf4-45cb-955e-e2c6e7b19ece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:43:55.945921 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9e376c-0bf4-45cb-955e-e2c6e7b19ece admin admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:43:55.947166 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9e376c-0bf4-45cb-955e-e2c6e7b19ece admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:43:55.947644 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9e376c-0bf4-45cb-955e-e2c6e7b19ece admin admin] PATH_INFO: `/v3/projects/3775373d9a9f4f92bdb217d2017f7aec/users/866e4647336249249942f9ed1a688c02/roles/66c1074780954f8e87ed11a5c4e8c36a` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:43:55.954142 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1382/2768] 10.222.0.64 () {60 vars in 1153 bytes} [Tue Aug 2 10:43:55 2022] POST /identity/v3/projects => generated 371 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:43:55.959057 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df98d75-e8ea-40e6-ac93-d1956387a046 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:43:55.977088 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9e376c-0bf4-45cb-955e-e2c6e7b19ece admin admin] RBAC: Authorizing `identity:create_grant(project_id=3775373d9a9f4f92bdb217d2017f7aec, user_id=866e4647336249249942f9ed1a688c02, role_id=66c1074780954f8e87ed11a5c4e8c36a)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:43:55.986102 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9e376c-0bf4-45cb-955e-e2c6e7b19ece admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:43:56.013572 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1387/2769] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 10:43:55 2022] PUT /identity/v3/projects/3775373d9a9f4f92bdb217d2017f7aec/users/866e4647336249249942f9ed1a688c02/roles/66c1074780954f8e87ed11a5c4e8c36a => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:43:56.091136 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89997328-3fe7-46ca-a09d-a6f5933043fe None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:43:56.091429 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89997328-3fe7-46ca-a09d-a6f5933043fe None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:43:56.091710 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89997328-3fe7-46ca-a09d-a6f5933043fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:43:56.114128 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df98d75-e8ea-40e6-ac93-d1956387a046 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:43:56.139744 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-89997328-3fe7-46ca-a09d-a6f5933043fe None None] MFA Rules not processed for user `866e4647336249249942f9ed1a688c02`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:43:56.143156 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-89997328-3fe7-46ca-a09d-a6f5933043fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:43:56.297868 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2252f154-806f-499c-9ab2-a36861669566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:43:56.298578 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2252f154-806f-499c-9ab2-a36861669566 admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:43:56.298851 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2252f154-806f-499c-9ab2-a36861669566 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:43:56.299133 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2252f154-806f-499c-9ab2-a36861669566 admin admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:43:56.305603 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2252f154-806f-499c-9ab2-a36861669566 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:43:56.311020 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2252f154-806f-499c-9ab2-a36861669566 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:43:56.329761 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1388/2770] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:43:56 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:43:56.333991 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89997328-3fe7-46ca-a09d-a6f5933043fe None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:43:56.365856 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1383/2771] 10.222.0.64 () {60 vars in 1144 bytes} [Tue Aug 2 10:43:55 2022] POST /identity/v3/users => generated 292 bytes in 404 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:43:56.366244 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89997328-3fe7-46ca-a09d-a6f5933043fe None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:43:56.367450 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2252f154-806f-499c-9ab2-a36861669566 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:43:56.375705 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efecdf43-e3d3-4aeb-a35f-5833a075ab5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:43:56.377701 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efecdf43-e3d3-4aeb-a35f-5833a075ab5c admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:43:56.378591 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efecdf43-e3d3-4aeb-a35f-5833a075ab5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:43:56.379595 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efecdf43-e3d3-4aeb-a35f-5833a075ab5c admin admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:43:56.381360 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efecdf43-e3d3-4aeb-a35f-5833a075ab5c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:43:56.384374 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efecdf43-e3d3-4aeb-a35f-5833a075ab5c admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:43:56.395272 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2252f154-806f-499c-9ab2-a36861669566 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:43:56.402032 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76d6bd3-dfc0-4ff4-a44b-8ad87e90f8ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:43:56.402989 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76d6bd3-dfc0-4ff4-a44b-8ad87e90f8ba admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:43:56.403436 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76d6bd3-dfc0-4ff4-a44b-8ad87e90f8ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:43:56.404357 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76d6bd3-dfc0-4ff4-a44b-8ad87e90f8ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:43:56.405279 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76d6bd3-dfc0-4ff4-a44b-8ad87e90f8ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:43:56.407490 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76d6bd3-dfc0-4ff4-a44b-8ad87e90f8ba admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:43:56.421846 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1384/2772] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:43:56 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:43:56.427027 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76d6bd3-dfc0-4ff4-a44b-8ad87e90f8ba admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:43:56.440910 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76d6bd3-dfc0-4ff4-a44b-8ad87e90f8ba admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:43:56.449099 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3d02b6-c517-42c5-9eb0-f4e700bda979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:43:56.449997 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3d02b6-c517-42c5-9eb0-f4e700bda979 admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:43:56.450288 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3d02b6-c517-42c5-9eb0-f4e700bda979 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:43:56.450564 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3d02b6-c517-42c5-9eb0-f4e700bda979 admin admin] PATH_INFO: `/v3/projects/bbb19e16311a4aecac84d389a602e4ad/users/2dce115b92204af5aeacba8b4257ae7c/roles/1000cf318fa14915b8091599eaead5de` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:43:56.452976 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1389/2773] 10.222.0.64 () {60 vars in 1153 bytes} [Tue Aug 2 10:43:56 2022] POST /identity/v3/projects => generated 377 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:43:56.457446 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efecdf43-e3d3-4aeb-a35f-5833a075ab5c admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:43:56.480540 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3d02b6-c517-42c5-9eb0-f4e700bda979 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbb19e16311a4aecac84d389a602e4ad, user_id=2dce115b92204af5aeacba8b4257ae7c, role_id=1000cf318fa14915b8091599eaead5de)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:43:56.484922 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3d02b6-c517-42c5-9eb0-f4e700bda979 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:43:56.508039 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1385/2774] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 10:43:56 2022] PUT /identity/v3/projects/bbb19e16311a4aecac84d389a602e4ad/users/2dce115b92204af5aeacba8b4257ae7c/roles/1000cf318fa14915b8091599eaead5de => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:43:56.514257 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3d02b6-c517-42c5-9eb0-f4e700bda979 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:43:56.584820 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:43:56.617295 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efecdf43-e3d3-4aeb-a35f-5833a075ab5c admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:43:56.662187 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3d02b6-c517-42c5-9eb0-f4e700bda979 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:43:56.665127 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:43:56.671080 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893fd497-66a8-4b8a-8da1-8a316357e011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:43:56.672006 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893fd497-66a8-4b8a-8da1-8a316357e011 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:43:56.672317 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893fd497-66a8-4b8a-8da1-8a316357e011 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:43:56.672578 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893fd497-66a8-4b8a-8da1-8a316357e011 admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:43:56.673294 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893fd497-66a8-4b8a-8da1-8a316357e011 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:43:56.676743 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893fd497-66a8-4b8a-8da1-8a316357e011 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:43:56.701672 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1386/2775] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:43:56 2022] GET /identity/v3/roles => generated 1420 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:43:56.705704 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893fd497-66a8-4b8a-8da1-8a316357e011 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:43:56.726593 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893fd497-66a8-4b8a-8da1-8a316357e011 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:43:56.736570 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2545be-8e3e-4a37-b644-aa230524f7ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:43:56.737503 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2545be-8e3e-4a37-b644-aa230524f7ab admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:43:56.737906 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2545be-8e3e-4a37-b644-aa230524f7ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:43:56.738752 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2545be-8e3e-4a37-b644-aa230524f7ab admin admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:43:56.739127 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2545be-8e3e-4a37-b644-aa230524f7ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:43:56.739583 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1cee45-dbca-4b43-b61e-1298f16dc8ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:43:56.740553 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1cee45-dbca-4b43-b61e-1298f16dc8ce admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:43:56.741024 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1cee45-dbca-4b43-b61e-1298f16dc8ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:43:56.741475 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1cee45-dbca-4b43-b61e-1298f16dc8ce admin admin] PATH_INFO: `/v3/projects/bbb19e16311a4aecac84d389a602e4ad/users/2dce115b92204af5aeacba8b4257ae7c/roles/1000cf318fa14915b8091599eaead5de` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:43:56.741658 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2545be-8e3e-4a37-b644-aa230524f7ab admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:43:56.745528 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1cee45-dbca-4b43-b61e-1298f16dc8ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbb19e16311a4aecac84d389a602e4ad, user_id=2dce115b92204af5aeacba8b4257ae7c, role_id=1000cf318fa14915b8091599eaead5de)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:43:56.749945 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1cee45-dbca-4b43-b61e-1298f16dc8ce admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:43:56.768334 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1387/2776] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 10:43:56 2022] PUT /identity/v3/projects/bbb19e16311a4aecac84d389a602e4ad/users/2dce115b92204af5aeacba8b4257ae7c/roles/1000cf318fa14915b8091599eaead5de => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:43:56.781669 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1390/2777] 10.222.0.64 () {60 vars in 1144 bytes} [Tue Aug 2 10:43:56 2022] POST /identity/v3/users => generated 289 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:43:56.782688 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1cee45-dbca-4b43-b61e-1298f16dc8ce admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:43:56.938510 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1cee45-dbca-4b43-b61e-1298f16dc8ce admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:43:56.946018 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4169c9b5-c4b6-4438-8ff0-516ccf7a1bca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:43:56.947064 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4169c9b5-c4b6-4438-8ff0-516ccf7a1bca admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:43:56.947569 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4169c9b5-c4b6-4438-8ff0-516ccf7a1bca admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:43:56.948605 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4169c9b5-c4b6-4438-8ff0-516ccf7a1bca admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:43:56.949684 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4169c9b5-c4b6-4438-8ff0-516ccf7a1bca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:43:56.951884 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4169c9b5-c4b6-4438-8ff0-516ccf7a1bca admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:43:56.968307 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1388/2778] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:43:56 2022] GET /identity/v3/roles => generated 1420 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:43:56.971976 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2545be-8e3e-4a37-b644-aa230524f7ab admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:43:56.987662 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2545be-8e3e-4a37-b644-aa230524f7ab admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:43:56.996876 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f90fe6-09d8-4205-b4bb-bed560a95ce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:43:56.997745 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f90fe6-09d8-4205-b4bb-bed560a95ce9 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:43:56.998088 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f90fe6-09d8-4205-b4bb-bed560a95ce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:43:56.998369 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f90fe6-09d8-4205-b4bb-bed560a95ce9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:43:56.999311 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f90fe6-09d8-4205-b4bb-bed560a95ce9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:43:57.001035 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f90fe6-09d8-4205-b4bb-bed560a95ce9 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:43:57.029409 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1391/2779] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:43:56 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:43:57.029998 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4169c9b5-c4b6-4438-8ff0-516ccf7a1bca admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:43:57.050794 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4169c9b5-c4b6-4438-8ff0-516ccf7a1bca admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:43:57.056873 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a3b920-4187-4d9a-a815-8e7016869af6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:43:57.057956 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a3b920-4187-4d9a-a815-8e7016869af6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:43:57.057956 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a3b920-4187-4d9a-a815-8e7016869af6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:43:57.058441 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a3b920-4187-4d9a-a815-8e7016869af6 admin admin] PATH_INFO: `/v3/projects/d536c786011c480e87e1dd8c9e0e194f/users/034079393cb94d1780a03872b65ddd03/roles/66c1074780954f8e87ed11a5c4e8c36a` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:43:57.092665 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a3b920-4187-4d9a-a815-8e7016869af6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d536c786011c480e87e1dd8c9e0e194f, user_id=034079393cb94d1780a03872b65ddd03, role_id=66c1074780954f8e87ed11a5c4e8c36a)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:43:57.095396 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a3b920-4187-4d9a-a815-8e7016869af6 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:43:57.130876 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1389/2780] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 10:43:57 2022] PUT /identity/v3/projects/d536c786011c480e87e1dd8c9e0e194f/users/034079393cb94d1780a03872b65ddd03/roles/66c1074780954f8e87ed11a5c4e8c36a => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:43:57.133010 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d099f809-18da-426d-b932-04a359cbfff4 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:43:57.133384 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d099f809-18da-426d-b932-04a359cbfff4 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:43:57.134235 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d099f809-18da-426d-b932-04a359cbfff4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:43:57.173745 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-d099f809-18da-426d-b932-04a359cbfff4 None None] MFA Rules not processed for user `034079393cb94d1780a03872b65ddd03`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:43:57.175567 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-d099f809-18da-426d-b932-04a359cbfff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:43:57.326014 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1392/2781] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:43:57 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:43:57.330835 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a3b920-4187-4d9a-a815-8e7016869af6 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:43:57.477201 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a3b920-4187-4d9a-a815-8e7016869af6 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:43:57.484160 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbff07f6-72c1-4702-830a-50b48844ecc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:43:57.484929 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbff07f6-72c1-4702-830a-50b48844ecc8 admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:43:57.485216 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbff07f6-72c1-4702-830a-50b48844ecc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:43:57.485529 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbff07f6-72c1-4702-830a-50b48844ecc8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:43:57.486334 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbff07f6-72c1-4702-830a-50b48844ecc8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:43:57.488497 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbff07f6-72c1-4702-830a-50b48844ecc8 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:43:57.527027 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1390/2782] 10.222.0.64 () {60 vars in 1153 bytes} [Tue Aug 2 10:43:57 2022] POST /identity/v3/projects => generated 377 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:43:57.528276 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d099f809-18da-426d-b932-04a359cbfff4 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:43:57.542991 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d099f809-18da-426d-b932-04a359cbfff4 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:43:57.660937 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a3a402-587e-4c60-846d-15e2539fc497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:43:57.661656 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a3a402-587e-4c60-846d-15e2539fc497 admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:43:57.662017 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a3a402-587e-4c60-846d-15e2539fc497 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:43:57.662370 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a3a402-587e-4c60-846d-15e2539fc497 admin admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:43:57.663276 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a3a402-587e-4c60-846d-15e2539fc497 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:43:57.667848 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a3a402-587e-4c60-846d-15e2539fc497 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:43:57.715292 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbff07f6-72c1-4702-830a-50b48844ecc8 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:43:57.715991 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1393/2783] 10.222.0.64 () {60 vars in 1144 bytes} [Tue Aug 2 10:43:57 2022] POST /identity/v3/users => generated 295 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:43:57.733698 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbff07f6-72c1-4702-830a-50b48844ecc8 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:43:57.746326 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fae02b-6f10-4308-baff-5e829052a006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:43:57.747281 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fae02b-6f10-4308-baff-5e829052a006 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:43:57.747821 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fae02b-6f10-4308-baff-5e829052a006 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:43:57.748990 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fae02b-6f10-4308-baff-5e829052a006 admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:43:57.750091 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fae02b-6f10-4308-baff-5e829052a006 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:43:57.752920 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fae02b-6f10-4308-baff-5e829052a006 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:43:57.767082 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:43:57.776683 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1391/2784] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:43:57 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:43:57.781518 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a3a402-587e-4c60-846d-15e2539fc497 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:43:57.797031 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a3a402-587e-4c60-846d-15e2539fc497 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:43:57.802081 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f87c88-15dd-42f3-a9fd-aea75ff690cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:43:57.803159 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f87c88-15dd-42f3-a9fd-aea75ff690cb admin admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:43:57.803577 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f87c88-15dd-42f3-a9fd-aea75ff690cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:43:57.803949 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f87c88-15dd-42f3-a9fd-aea75ff690cb admin admin] PATH_INFO: `/v3/projects/ef9429f2fe2b46b38d016f43c99e26fb/users/2e6b08f1ebf64329a1beb4b9bc8154bb/roles/1000cf318fa14915b8091599eaead5de` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:43:57.829203 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f87c88-15dd-42f3-a9fd-aea75ff690cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef9429f2fe2b46b38d016f43c99e26fb, user_id=2e6b08f1ebf64329a1beb4b9bc8154bb, role_id=1000cf318fa14915b8091599eaead5de)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:43:57.836222 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f87c88-15dd-42f3-a9fd-aea75ff690cb admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:43:57.842116 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:43:57.871013 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1394/2785] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 10:43:57 2022] PUT /identity/v3/projects/ef9429f2fe2b46b38d016f43c99e26fb/users/2e6b08f1ebf64329a1beb4b9bc8154bb/roles/1000cf318fa14915b8091599eaead5de => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:43:57.875591 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fae02b-6f10-4308-baff-5e829052a006 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:43:57.889270 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fae02b-6f10-4308-baff-5e829052a006 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:43:57.986714 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2074c580-8d12-4f07-bd2d-6b15bcd667b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:43:57.987568 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2074c580-8d12-4f07-bd2d-6b15bcd667b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:43:57.987953 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2074c580-8d12-4f07-bd2d-6b15bcd667b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:43:57.988529 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2074c580-8d12-4f07-bd2d-6b15bcd667b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:43:57.989526 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2074c580-8d12-4f07-bd2d-6b15bcd667b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:43:57.991744 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2074c580-8d12-4f07-bd2d-6b15bcd667b9 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:43:58.009713 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1392/2786] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:43:57 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:43:58.014036 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f87c88-15dd-42f3-a9fd-aea75ff690cb admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:43:58.067432 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f87c88-15dd-42f3-a9fd-aea75ff690cb admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:43:58.088124 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337d0b98-f92c-4b63-8a21-11d637189a01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:43:58.088124 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337d0b98-f92c-4b63-8a21-11d637189a01 admin admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:43:58.088124 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337d0b98-f92c-4b63-8a21-11d637189a01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:43:58.107296 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337d0b98-f92c-4b63-8a21-11d637189a01 admin admin] PATH_INFO: `/v3/projects/ef9429f2fe2b46b38d016f43c99e26fb/users/2e6b08f1ebf64329a1beb4b9bc8154bb/roles/1000cf318fa14915b8091599eaead5de` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:43:58.110575 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337d0b98-f92c-4b63-8a21-11d637189a01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef9429f2fe2b46b38d016f43c99e26fb, user_id=2e6b08f1ebf64329a1beb4b9bc8154bb, role_id=1000cf318fa14915b8091599eaead5de)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:43:58.114327 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337d0b98-f92c-4b63-8a21-11d637189a01 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:43:58.187713 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1395/2787] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 10:43:58 2022] PUT /identity/v3/projects/ef9429f2fe2b46b38d016f43c99e26fb/users/2e6b08f1ebf64329a1beb4b9bc8154bb/roles/1000cf318fa14915b8091599eaead5de => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:43:58.200445 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0c8152-4ab5-447c-9423-2f84341bd044 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:43:58.201332 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0c8152-4ab5-447c-9423-2f84341bd044 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:43:58.201773 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0c8152-4ab5-447c-9423-2f84341bd044 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:43:58.324903 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-db0c8152-4ab5-447c-9423-2f84341bd044 None None] MFA Rules not processed for user `2e6b08f1ebf64329a1beb4b9bc8154bb`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:43:58.327145 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-db0c8152-4ab5-447c-9423-2f84341bd044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:43:58.464670 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1393/2788] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:43:58 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:43:58.495403 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337d0b98-f92c-4b63-8a21-11d637189a01 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:43:58.497296 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-337d0b98-f92c-4b63-8a21-11d637189a01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:43:58.655146 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337d0b98-f92c-4b63-8a21-11d637189a01 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:43:58.660971 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca91a1a-229d-485f-9e6d-14392ed73946 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d7de62b9ee4039bdc79627ac7fb499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:43:58.661757 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca91a1a-229d-485f-9e6d-14392ed73946 service cinder] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:43:58.662052 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca91a1a-229d-485f-9e6d-14392ed73946 service cinder] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:43:58.662449 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca91a1a-229d-485f-9e6d-14392ed73946 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:43:58.685475 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca91a1a-229d-485f-9e6d-14392ed73946 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:43:58.687733 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca91a1a-229d-485f-9e6d-14392ed73946 service cinder] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:43:58.717042 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1396/2789] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:43:58 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:43:58.792239 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0c8152-4ab5-447c-9423-2f84341bd044 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:43:58.794309 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-db0c8152-4ab5-447c-9423-2f84341bd044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:43:58.871351 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0c8152-4ab5-447c-9423-2f84341bd044 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:43:58.903180 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bfebdc-6a75-42c9-9d65-2ed9059901b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d7de62b9ee4039bdc79627ac7fb499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:43:58.910569 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bfebdc-6a75-42c9-9d65-2ed9059901b7 service cinder] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:43:58.910943 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bfebdc-6a75-42c9-9d65-2ed9059901b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:43:58.911213 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bfebdc-6a75-42c9-9d65-2ed9059901b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:43:59.066247 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bfebdc-6a75-42c9-9d65-2ed9059901b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:43:59.068813 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bfebdc-6a75-42c9-9d65-2ed9059901b7 service cinder] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:43:59.086145 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1394/2790] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:43:58 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:43:59.170082 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca91a1a-229d-485f-9e6d-14392ed73946 service cinder] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:43:59.171120 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bfebdc-6a75-42c9-9d65-2ed9059901b7 service cinder] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:43:59.173367 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-3ca91a1a-229d-485f-9e6d-14392ed73946 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:43:59.174736 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-74bfebdc-6a75-42c9-9d65-2ed9059901b7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:43:59.265379 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:43:59.337889 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca91a1a-229d-485f-9e6d-14392ed73946 service cinder] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:43:59.348908 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d871ecdf-9cc1-4e6a-b204-46b87e9a9741 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b770f073c86b4d88837a8b1f796a8b1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:43:59.351186 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d871ecdf-9cc1-4e6a-b204-46b87e9a9741 service placement] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:43:59.351628 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d871ecdf-9cc1-4e6a-b204-46b87e9a9741 service placement] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:43:59.352037 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d871ecdf-9cc1-4e6a-b204-46b87e9a9741 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:43:59.355523 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-d871ecdf-9cc1-4e6a-b204-46b87e9a9741 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:43:59.384540 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d871ecdf-9cc1-4e6a-b204-46b87e9a9741 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:43:59.390506 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d871ecdf-9cc1-4e6a-b204-46b87e9a9741 service placement] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:43:59.396838 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bfebdc-6a75-42c9-9d65-2ed9059901b7 service cinder] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:43:59.410634 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1397/2791] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:43:59 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:43:59.412863 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6369a993-f703-4972-a2ed-adf1329a4774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:43:59.413726 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6369a993-f703-4972-a2ed-adf1329a4774 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:43:59.414203 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6369a993-f703-4972-a2ed-adf1329a4774 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:43:59.414698 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6369a993-f703-4972-a2ed-adf1329a4774 admin admin] PATH_INFO: `/v3/users/dfe32e33711e42118f3ff5021762604f` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:43:59.453999 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6369a993-f703-4972-a2ed-adf1329a4774 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dfe32e33711e42118f3ff5021762604f)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:43:59.456449 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6369a993-f703-4972-a2ed-adf1329a4774 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:43:59.589187 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-6369a993-f703-4972-a2ed-adf1329a4774 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dfe32e33711e42118f3ff5021762604f'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:43:59.607202 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-6369a993-f703-4972-a2ed-adf1329a4774 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dfe32e33711e42118f3ff5021762604f'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:43:59.612537 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-6369a993-f703-4972-a2ed-adf1329a4774 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dfe32e33711e42118f3ff5021762604f'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:43:59.633766 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-6369a993-f703-4972-a2ed-adf1329a4774 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dfe32e33711e42118f3ff5021762604f'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:43:59.664812 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-6369a993-f703-4972-a2ed-adf1329a4774 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dfe32e33711e42118f3ff5021762604f'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:43:59.677838 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1395/2792] 10.222.0.64 () {58 vars in 1224 bytes} [Tue Aug 2 10:43:59 2022] DELETE /identity/v3/users/dfe32e33711e42118f3ff5021762604f => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:44:00.074509 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d871ecdf-9cc1-4e6a-b204-46b87e9a9741 service placement] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:00.102999 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d871ecdf-9cc1-4e6a-b204-46b87e9a9741 service placement] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:00.181640 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c728bc-e83c-4c9b-bd2d-314993319802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:00.182581 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c728bc-e83c-4c9b-bd2d-314993319802 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:00.183160 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c728bc-e83c-4c9b-bd2d-314993319802 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:00.183599 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c728bc-e83c-4c9b-bd2d-314993319802 admin admin] PATH_INFO: `/v3/projects/d1f5fe53a76440268bbb5b72e551efd9` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:00.198531 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c728bc-e83c-4c9b-bd2d-314993319802 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1f5fe53a76440268bbb5b72e551efd9)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:00.201740 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c728bc-e83c-4c9b-bd2d-314993319802 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:00.223189 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:44:00.292971 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-44c728bc-e83c-4c9b-bd2d-314993319802 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1f5fe53a76440268bbb5b72e551efd9', 'request_id': 'req-44c728bc-e83c-4c9b-bd2d-314993319802'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:00.301347 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-44c728bc-e83c-4c9b-bd2d-314993319802 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1f5fe53a76440268bbb5b72e551efd9', 'request_id': 'req-44c728bc-e83c-4c9b-bd2d-314993319802'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:00.309861 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-44c728bc-e83c-4c9b-bd2d-314993319802 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1f5fe53a76440268bbb5b72e551efd9', 'request_id': 'req-44c728bc-e83c-4c9b-bd2d-314993319802'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:00.327967 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-44c728bc-e83c-4c9b-bd2d-314993319802 admin admin] The token cache is being invalidate because project d1f5fe53a76440268bbb5b72e551efd9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80153) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:44:00.328593 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-44c728bc-e83c-4c9b-bd2d-314993319802 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:00.333227 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-44c728bc-e83c-4c9b-bd2d-314993319802 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:00.335821 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1398/2793] 10.222.0.64 () {58 vars in 1233 bytes} [Tue Aug 2 10:44:00 2022] DELETE /identity/v3/projects/d1f5fe53a76440268bbb5b72e551efd9 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:44:00.342784 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39680784-f4bf-4561-8696-1c2baa782e2e None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:00.343376 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39680784-f4bf-4561-8696-1c2baa782e2e None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:00.343376 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39680784-f4bf-4561-8696-1c2baa782e2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:00.366079 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-39680784-f4bf-4561-8696-1c2baa782e2e None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:44:00.367401 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-39680784-f4bf-4561-8696-1c2baa782e2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:44:00.454135 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1396/2794] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:44:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:44:00.463355 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea69f8be-a350-4179-86b0-f2fca8f60da1 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:00.466773 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea69f8be-a350-4179-86b0-f2fca8f60da1 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:00.466773 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea69f8be-a350-4179-86b0-f2fca8f60da1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:00.493629 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-ea69f8be-a350-4179-86b0-f2fca8f60da1 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:44:00.495252 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-ea69f8be-a350-4179-86b0-f2fca8f60da1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:44:00.504848 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1399/2795] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:44:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:44:00.508514 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39680784-f4bf-4561-8696-1c2baa782e2e None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:00.521792 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39680784-f4bf-4561-8696-1c2baa782e2e None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:00.532952 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b32179-9667-4ab1-a63b-48ac3f88e6fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:00.533745 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b32179-9667-4ab1-a63b-48ac3f88e6fa admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:00.534133 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b32179-9667-4ab1-a63b-48ac3f88e6fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:00.534509 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b32179-9667-4ab1-a63b-48ac3f88e6fa admin admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:00.535334 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b32179-9667-4ab1-a63b-48ac3f88e6fa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:44:00.535977 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b32179-9667-4ab1-a63b-48ac3f88e6fa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:00.538199 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b32179-9667-4ab1-a63b-48ac3f88e6fa admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:00.559120 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1397/2796] 10.222.0.64 () {58 vars in 1153 bytes} [Tue Aug 2 10:44:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:44:00.560454 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea69f8be-a350-4179-86b0-f2fca8f60da1 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:00.572451 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea69f8be-a350-4179-86b0-f2fca8f60da1 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:00.589278 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc78cde-ac20-4055-ac13-77ef421538b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:00.591192 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc78cde-ac20-4055-ac13-77ef421538b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:00.591591 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc78cde-ac20-4055-ac13-77ef421538b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:00.591993 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc78cde-ac20-4055-ac13-77ef421538b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:00.593683 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc78cde-ac20-4055-ac13-77ef421538b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:00.600348 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc78cde-ac20-4055-ac13-77ef421538b5 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:00.683972 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1400/2797] 10.222.0.64 () {60 vars in 1153 bytes} [Tue Aug 2 10:44:00 2022] POST /identity/v3/projects => generated 379 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:44:00.685708 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b32179-9667-4ab1-a63b-48ac3f88e6fa admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:00.757513 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b32179-9667-4ab1-a63b-48ac3f88e6fa admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:00.787009 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:44:00.901092 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20608a6e-d2da-4f57-8c4d-14bf17e27e54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:00.903009 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20608a6e-d2da-4f57-8c4d-14bf17e27e54 admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:00.903473 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20608a6e-d2da-4f57-8c4d-14bf17e27e54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:00.903915 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20608a6e-d2da-4f57-8c4d-14bf17e27e54 admin admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:00.905087 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20608a6e-d2da-4f57-8c4d-14bf17e27e54 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:00.907650 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20608a6e-d2da-4f57-8c4d-14bf17e27e54 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:00.941607 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1398/2798] 10.222.0.64 () {60 vars in 1144 bytes} [Tue Aug 2 10:44:00 2022] POST /identity/v3/users => generated 290 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:44:00.960632 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc78cde-ac20-4055-ac13-77ef421538b5 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:00.977247 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc78cde-ac20-4055-ac13-77ef421538b5 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:01.005858 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b178c181-21b1-431f-a04d-7881d858c7c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:01.006426 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b178c181-21b1-431f-a04d-7881d858c7c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:01.006721 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b178c181-21b1-431f-a04d-7881d858c7c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:01.007001 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b178c181-21b1-431f-a04d-7881d858c7c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:01.012037 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b178c181-21b1-431f-a04d-7881d858c7c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:01.013780 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b178c181-21b1-431f-a04d-7881d858c7c5 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:01.050373 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1401/2799] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:44:00 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:44:01.063143 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20608a6e-d2da-4f57-8c4d-14bf17e27e54 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:01.083049 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20608a6e-d2da-4f57-8c4d-14bf17e27e54 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:01.107373 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251fd813-5c65-4467-bbfd-68f452992f6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:01.108834 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251fd813-5c65-4467-bbfd-68f452992f6e admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:01.109387 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251fd813-5c65-4467-bbfd-68f452992f6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:01.109913 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251fd813-5c65-4467-bbfd-68f452992f6e admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:01.110889 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251fd813-5c65-4467-bbfd-68f452992f6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:01.113066 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251fd813-5c65-4467-bbfd-68f452992f6e admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:01.131853 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b178c181-21b1-431f-a04d-7881d858c7c5 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:01.132536 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1399/2800] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:44:01 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:44:01.151098 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b178c181-21b1-431f-a04d-7881d858c7c5 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:01.168133 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7b3f0c-1f41-4c59-a664-8ab9327e0dee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:01.169206 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7b3f0c-1f41-4c59-a664-8ab9327e0dee admin admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:01.169724 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7b3f0c-1f41-4c59-a664-8ab9327e0dee admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:01.170377 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7b3f0c-1f41-4c59-a664-8ab9327e0dee admin admin] PATH_INFO: `/v3/projects/801dc2387bda43cb905f9e8e6c5469c7/users/3342fcfdb30345faabdf616a332daacd/roles/66c1074780954f8e87ed11a5c4e8c36a` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:01.198828 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7b3f0c-1f41-4c59-a664-8ab9327e0dee admin admin] RBAC: Authorizing `identity:create_grant(project_id=801dc2387bda43cb905f9e8e6c5469c7, user_id=3342fcfdb30345faabdf616a332daacd, role_id=66c1074780954f8e87ed11a5c4e8c36a)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:01.204388 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7b3f0c-1f41-4c59-a664-8ab9327e0dee admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:01.273944 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1402/2801] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 10:44:01 2022] PUT /identity/v3/projects/801dc2387bda43cb905f9e8e6c5469c7/users/3342fcfdb30345faabdf616a332daacd/roles/66c1074780954f8e87ed11a5c4e8c36a => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:44:01.286610 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5b79c3-5ea6-4b41-bb84-8df6d574dcfe None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:01.287077 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5b79c3-5ea6-4b41-bb84-8df6d574dcfe None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:01.287433 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5b79c3-5ea6-4b41-bb84-8df6d574dcfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:01.363802 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-ae5b79c3-5ea6-4b41-bb84-8df6d574dcfe None None] MFA Rules not processed for user `3342fcfdb30345faabdf616a332daacd`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:44:01.365457 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-ae5b79c3-5ea6-4b41-bb84-8df6d574dcfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:44:01.544606 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1400/2802] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:44:01 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:44:01.545836 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7b3f0c-1f41-4c59-a664-8ab9327e0dee admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:01.776127 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7b3f0c-1f41-4c59-a664-8ab9327e0dee admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:01.804010 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b391cac-f101-4dea-8add-753e0b3ff1e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:01.804968 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b391cac-f101-4dea-8add-753e0b3ff1e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:01.805122 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b391cac-f101-4dea-8add-753e0b3ff1e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:01.805303 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b391cac-f101-4dea-8add-753e0b3ff1e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:01.806558 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b391cac-f101-4dea-8add-753e0b3ff1e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:01.813218 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b391cac-f101-4dea-8add-753e0b3ff1e6 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:01.870228 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1403/2803] 10.222.0.64 () {60 vars in 1153 bytes} [Tue Aug 2 10:44:01 2022] POST /identity/v3/projects => generated 381 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:44:01.874886 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5b79c3-5ea6-4b41-bb84-8df6d574dcfe None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:01.895151 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5b79c3-5ea6-4b41-bb84-8df6d574dcfe None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:02.079197 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be742434-ea33-4148-9ab9-724f5f7d28d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:02.083941 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be742434-ea33-4148-9ab9-724f5f7d28d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:02.084458 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be742434-ea33-4148-9ab9-724f5f7d28d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:02.084837 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be742434-ea33-4148-9ab9-724f5f7d28d3 admin admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:02.086328 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be742434-ea33-4148-9ab9-724f5f7d28d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:02.088815 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be742434-ea33-4148-9ab9-724f5f7d28d3 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:02.169735 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1401/2804] 10.222.0.64 () {60 vars in 1144 bytes} [Tue Aug 2 10:44:01 2022] POST /identity/v3/users => generated 297 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:44:02.174264 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b391cac-f101-4dea-8add-753e0b3ff1e6 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:02.194835 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b391cac-f101-4dea-8add-753e0b3ff1e6 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:02.203514 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac27e48-3f57-4d02-897a-654a1c743610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:02.204621 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac27e48-3f57-4d02-897a-654a1c743610 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:02.205148 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac27e48-3f57-4d02-897a-654a1c743610 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:02.205716 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac27e48-3f57-4d02-897a-654a1c743610 admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:02.206837 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac27e48-3f57-4d02-897a-654a1c743610 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:02.210530 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac27e48-3f57-4d02-897a-654a1c743610 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:02.231517 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1404/2805] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:44:02 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:44:02.237131 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be742434-ea33-4148-9ab9-724f5f7d28d3 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:02.252680 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be742434-ea33-4148-9ab9-724f5f7d28d3 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:02.263376 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03efd8c2-58b3-4735-954c-afeb75e2268e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:02.265005 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03efd8c2-58b3-4735-954c-afeb75e2268e admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:02.265188 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03efd8c2-58b3-4735-954c-afeb75e2268e admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:02.265639 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03efd8c2-58b3-4735-954c-afeb75e2268e admin admin] PATH_INFO: `/v3/projects/c169bbd936994784873efe6ec7b15ca7/users/502828022f0647b4805168cd977dbfbb/roles/1000cf318fa14915b8091599eaead5de` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:02.290414 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:44:02.305113 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03efd8c2-58b3-4735-954c-afeb75e2268e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c169bbd936994784873efe6ec7b15ca7, user_id=502828022f0647b4805168cd977dbfbb, role_id=1000cf318fa14915b8091599eaead5de)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:02.310130 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03efd8c2-58b3-4735-954c-afeb75e2268e admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:02.358795 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac27e48-3f57-4d02-897a-654a1c743610 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:02.359213 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1402/2806] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 10:44:02 2022] PUT /identity/v3/projects/c169bbd936994784873efe6ec7b15ca7/users/502828022f0647b4805168cd977dbfbb/roles/1000cf318fa14915b8091599eaead5de => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:44:02.396018 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac27e48-3f57-4d02-897a-654a1c743610 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:02.402041 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:44:02.517972 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d10527b-71c3-43f7-a5ca-55cb7a61111c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:02.520801 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d10527b-71c3-43f7-a5ca-55cb7a61111c admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:02.521411 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d10527b-71c3-43f7-a5ca-55cb7a61111c admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:02.522020 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d10527b-71c3-43f7-a5ca-55cb7a61111c admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:02.525331 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d10527b-71c3-43f7-a5ca-55cb7a61111c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:02.528601 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d10527b-71c3-43f7-a5ca-55cb7a61111c admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:02.549126 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1405/2807] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:44:02 2022] GET /identity/v3/roles => generated 1420 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:44:02.554556 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03efd8c2-58b3-4735-954c-afeb75e2268e admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:02.581752 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03efd8c2-58b3-4735-954c-afeb75e2268e admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:02.590280 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4332655-b130-4225-a012-7fde8b1fe025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:02.591161 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4332655-b130-4225-a012-7fde8b1fe025 admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:02.591573 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4332655-b130-4225-a012-7fde8b1fe025 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:02.592006 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4332655-b130-4225-a012-7fde8b1fe025 admin admin] PATH_INFO: `/v3/projects/c169bbd936994784873efe6ec7b15ca7/users/502828022f0647b4805168cd977dbfbb/roles/1000cf318fa14915b8091599eaead5de` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:02.595636 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4332655-b130-4225-a012-7fde8b1fe025 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c169bbd936994784873efe6ec7b15ca7, user_id=502828022f0647b4805168cd977dbfbb, role_id=1000cf318fa14915b8091599eaead5de)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:02.601010 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4332655-b130-4225-a012-7fde8b1fe025 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:02.638700 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1403/2808] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 10:44:02 2022] PUT /identity/v3/projects/c169bbd936994784873efe6ec7b15ca7/users/502828022f0647b4805168cd977dbfbb/roles/1000cf318fa14915b8091599eaead5de => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:44:02.659036 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bf789c-7b91-4aa8-864b-e8f0cf7d201a None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:02.659346 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bf789c-7b91-4aa8-864b-e8f0cf7d201a None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:02.659632 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bf789c-7b91-4aa8-864b-e8f0cf7d201a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:02.746020 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-60bf789c-7b91-4aa8-864b-e8f0cf7d201a None None] MFA Rules not processed for user `502828022f0647b4805168cd977dbfbb`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:44:02.747282 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-60bf789c-7b91-4aa8-864b-e8f0cf7d201a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:44:02.955114 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1406/2809] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:44:02 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:44:02.971023 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4332655-b130-4225-a012-7fde8b1fe025 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:02.974172 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-e4332655-b130-4225-a012-7fde8b1fe025 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:44:03.104202 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4332655-b130-4225-a012-7fde8b1fe025 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:03.120646 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110dd363-a2e4-4338-b3f7-748a1981f229 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d7de62b9ee4039bdc79627ac7fb499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:03.121310 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110dd363-a2e4-4338-b3f7-748a1981f229 service cinder] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:03.121619 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110dd363-a2e4-4338-b3f7-748a1981f229 service cinder] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:03.121915 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110dd363-a2e4-4338-b3f7-748a1981f229 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:03.144458 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110dd363-a2e4-4338-b3f7-748a1981f229 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:03.145789 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110dd363-a2e4-4338-b3f7-748a1981f229 service cinder] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:03.178411 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1404/2810] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:44:02 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:44:03.213212 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d40de8-4b35-46ec-8216-a1f4f4ecff2c None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:03.214761 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d40de8-4b35-46ec-8216-a1f4f4ecff2c None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:03.215234 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d40de8-4b35-46ec-8216-a1f4f4ecff2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:03.339727 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-b9d40de8-4b35-46ec-8216-a1f4f4ecff2c None None] MFA Rules not processed for user `2dce115b92204af5aeacba8b4257ae7c`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:44:03.341010 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-b9d40de8-4b35-46ec-8216-a1f4f4ecff2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:44:03.377199 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110dd363-a2e4-4338-b3f7-748a1981f229 service cinder] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:03.379302 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-110dd363-a2e4-4338-b3f7-748a1981f229 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:44:03.532512 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1407/2811] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:44:03 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:44:03.542364 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d40de8-4b35-46ec-8216-a1f4f4ecff2c None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:03.544595 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-b9d40de8-4b35-46ec-8216-a1f4f4ecff2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:44:03.575440 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110dd363-a2e4-4338-b3f7-748a1981f229 service cinder] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:03.590875 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:44:03.591183 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308743c6-a4bd-4be7-8960-e172d3b13f3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:03.591862 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308743c6-a4bd-4be7-8960-e172d3b13f3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:03.592149 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308743c6-a4bd-4be7-8960-e172d3b13f3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:03.594924 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308743c6-a4bd-4be7-8960-e172d3b13f3e admin admin] PATH_INFO: `/v3/users/866e4647336249249942f9ed1a688c02` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:03.600709 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308743c6-a4bd-4be7-8960-e172d3b13f3e admin admin] RBAC: Authorizing `identity:delete_user(user_id=866e4647336249249942f9ed1a688c02)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:03.605744 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308743c6-a4bd-4be7-8960-e172d3b13f3e admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:03.793361 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-308743c6-a4bd-4be7-8960-e172d3b13f3e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '866e4647336249249942f9ed1a688c02'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:03.815598 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-308743c6-a4bd-4be7-8960-e172d3b13f3e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '866e4647336249249942f9ed1a688c02'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:03.816712 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-308743c6-a4bd-4be7-8960-e172d3b13f3e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '866e4647336249249942f9ed1a688c02'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:03.838892 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-308743c6-a4bd-4be7-8960-e172d3b13f3e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '866e4647336249249942f9ed1a688c02'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:03.853426 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d40de8-4b35-46ec-8216-a1f4f4ecff2c None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:03.889315 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7af94e4-6858-4d8e-abbd-633e53db297d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99655e1f9ca24472a86cd26b0dbd1fc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:03.905799 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-308743c6-a4bd-4be7-8960-e172d3b13f3e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '866e4647336249249942f9ed1a688c02'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:03.906137 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7af94e4-6858-4d8e-abbd-633e53db297d service nova] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:03.906434 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7af94e4-6858-4d8e-abbd-633e53db297d service nova] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:03.906824 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7af94e4-6858-4d8e-abbd-633e53db297d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:03.908776 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:44:03.917675 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-f7af94e4-6858-4d8e-abbd-633e53db297d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:44:03.933273 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1405/2812] 10.222.0.64 () {58 vars in 1224 bytes} [Tue Aug 2 10:44:03 2022] DELETE /identity/v3/users/866e4647336249249942f9ed1a688c02 => generated 0 bytes in 554 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:44:03.935757 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4b9e43-3738-4d86-8ce6-3f12ab32169b None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:03.935964 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4b9e43-3738-4d86-8ce6-3f12ab32169b None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:03.936683 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4b9e43-3738-4d86-8ce6-3f12ab32169b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:04.000055 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-7f4b9e43-3738-4d86-8ce6-3f12ab32169b None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:44:04.001898 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-7f4b9e43-3738-4d86-8ce6-3f12ab32169b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:44:04.112667 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7af94e4-6858-4d8e-abbd-633e53db297d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:04.114318 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7af94e4-6858-4d8e-abbd-633e53db297d service nova] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:04.131285 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1406/2813] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:44:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:44:04.131285 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4b9e43-3738-4d86-8ce6-3f12ab32169b None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:04.134235 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-7f4b9e43-3738-4d86-8ce6-3f12ab32169b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:44:04.150332 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1408/2814] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:44:03 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 608 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:44:04.151260 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7af94e4-6858-4d8e-abbd-633e53db297d service nova] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:04.157667 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-f7af94e4-6858-4d8e-abbd-633e53db297d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:44:04.275815 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7af94e4-6858-4d8e-abbd-633e53db297d service nova] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:04.328342 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4b9e43-3738-4d86-8ce6-3f12ab32169b None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:04.341915 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093030db-9e06-4b60-98df-53c60072c8f5 tempest-VolumeQuotaClassesTest-1838313936 tempest-VolumeQuotaClassesTest-1838313936-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '502828022f0647b4805168cd977dbfbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c169bbd936994784873efe6ec7b15ca7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:04.347769 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093030db-9e06-4b60-98df-53c60072c8f5 tempest-VolumeQuotaClassesTest-1838313936 tempest-VolumeQuotaClassesTest-1838313936-project-admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:04.348324 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093030db-9e06-4b60-98df-53c60072c8f5 tempest-VolumeQuotaClassesTest-1838313936 tempest-VolumeQuotaClassesTest-1838313936-project-admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:04.348711 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093030db-9e06-4b60-98df-53c60072c8f5 tempest-VolumeQuotaClassesTest-1838313936 tempest-VolumeQuotaClassesTest-1838313936-project-admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:04.352622 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093030db-9e06-4b60-98df-53c60072c8f5 tempest-VolumeQuotaClassesTest-1838313936 tempest-VolumeQuotaClassesTest-1838313936-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:44:04.353099 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093030db-9e06-4b60-98df-53c60072c8f5 tempest-VolumeQuotaClassesTest-1838313936 tempest-VolumeQuotaClassesTest-1838313936-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:04.355025 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093030db-9e06-4b60-98df-53c60072c8f5 tempest-VolumeQuotaClassesTest-1838313936 tempest-VolumeQuotaClassesTest-1838313936-project-admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:04.401043 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1407/2815] 10.222.0.64 () {58 vars in 1153 bytes} [Tue Aug 2 10:44:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:44:04.402739 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093030db-9e06-4b60-98df-53c60072c8f5 tempest-VolumeQuotaClassesTest-1838313936 tempest-VolumeQuotaClassesTest-1838313936-project-admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:04.421700 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093030db-9e06-4b60-98df-53c60072c8f5 tempest-VolumeQuotaClassesTest-1838313936 tempest-VolumeQuotaClassesTest-1838313936-project-admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:04.422995 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5936ab49-60f6-4df5-814c-5ba34ae28065 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:04.424218 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5936ab49-60f6-4df5-814c-5ba34ae28065 service neutron] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:04.424673 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5936ab49-60f6-4df5-814c-5ba34ae28065 service neutron] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:04.424799 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5936ab49-60f6-4df5-814c-5ba34ae28065 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:04.429435 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01043efa-bba6-49e8-a32a-daddee20f645 tempest-VolumeQuotaClassesTest-1838313936 tempest-VolumeQuotaClassesTest-1838313936-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '502828022f0647b4805168cd977dbfbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c169bbd936994784873efe6ec7b15ca7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:04.431699 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01043efa-bba6-49e8-a32a-daddee20f645 tempest-VolumeQuotaClassesTest-1838313936 tempest-VolumeQuotaClassesTest-1838313936-project-admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:04.432337 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01043efa-bba6-49e8-a32a-daddee20f645 tempest-VolumeQuotaClassesTest-1838313936 tempest-VolumeQuotaClassesTest-1838313936-project-admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:04.433015 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01043efa-bba6-49e8-a32a-daddee20f645 tempest-VolumeQuotaClassesTest-1838313936 tempest-VolumeQuotaClassesTest-1838313936-project-admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:04.434296 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01043efa-bba6-49e8-a32a-daddee20f645 tempest-VolumeQuotaClassesTest-1838313936 tempest-VolumeQuotaClassesTest-1838313936-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:04.437302 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01043efa-bba6-49e8-a32a-daddee20f645 tempest-VolumeQuotaClassesTest-1838313936 tempest-VolumeQuotaClassesTest-1838313936-project-admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:04.445455 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5936ab49-60f6-4df5-814c-5ba34ae28065 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:04.448822 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5936ab49-60f6-4df5-814c-5ba34ae28065 service neutron] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:04.475634 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1409/2816] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:44:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:44:04.506565 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1408/2817] 10.222.0.64 () {60 vars in 1153 bytes} [Tue Aug 2 10:44:04 2022] POST /identity/v3/projects => generated 355 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:44:04.529075 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5936ab49-60f6-4df5-814c-5ba34ae28065 service neutron] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:04.566381 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5936ab49-60f6-4df5-814c-5ba34ae28065 service neutron] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:04.574101 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:44:04.666498 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a79d2ea-d8cf-42de-a58f-71f52a760858 tempest-VolumeQuotaClassesTest-1838313936 tempest-VolumeQuotaClassesTest-1838313936-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '502828022f0647b4805168cd977dbfbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c169bbd936994784873efe6ec7b15ca7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:04.666936 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a79d2ea-d8cf-42de-a58f-71f52a760858 tempest-VolumeQuotaClassesTest-1838313936 tempest-VolumeQuotaClassesTest-1838313936-project-admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:04.667311 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a79d2ea-d8cf-42de-a58f-71f52a760858 tempest-VolumeQuotaClassesTest-1838313936 tempest-VolumeQuotaClassesTest-1838313936-project-admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:04.667771 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a79d2ea-d8cf-42de-a58f-71f52a760858 tempest-VolumeQuotaClassesTest-1838313936 tempest-VolumeQuotaClassesTest-1838313936-project-admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:04.669146 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a79d2ea-d8cf-42de-a58f-71f52a760858 tempest-VolumeQuotaClassesTest-1838313936 tempest-VolumeQuotaClassesTest-1838313936-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:44:04.669308 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a79d2ea-d8cf-42de-a58f-71f52a760858 tempest-VolumeQuotaClassesTest-1838313936 tempest-VolumeQuotaClassesTest-1838313936-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:04.672154 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a79d2ea-d8cf-42de-a58f-71f52a760858 tempest-VolumeQuotaClassesTest-1838313936 tempest-VolumeQuotaClassesTest-1838313936-project-admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:04.708505 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1410/2818] 10.222.0.64 () {58 vars in 1153 bytes} [Tue Aug 2 10:44:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:44:04.798475 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01043efa-bba6-49e8-a32a-daddee20f645 tempest-VolumeQuotaClassesTest-1838313936 tempest-VolumeQuotaClassesTest-1838313936-project-admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:04.815613 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01043efa-bba6-49e8-a32a-daddee20f645 tempest-VolumeQuotaClassesTest-1838313936 tempest-VolumeQuotaClassesTest-1838313936-project-admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:04.820775 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944371e8-2882-426c-abe8-aa34b2ae5113 tempest-VolumeQuotaClassesTest-1838313936 tempest-VolumeQuotaClassesTest-1838313936-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '502828022f0647b4805168cd977dbfbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c169bbd936994784873efe6ec7b15ca7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:04.821476 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944371e8-2882-426c-abe8-aa34b2ae5113 tempest-VolumeQuotaClassesTest-1838313936 tempest-VolumeQuotaClassesTest-1838313936-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:04.821996 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944371e8-2882-426c-abe8-aa34b2ae5113 tempest-VolumeQuotaClassesTest-1838313936 tempest-VolumeQuotaClassesTest-1838313936-project-admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:04.822446 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944371e8-2882-426c-abe8-aa34b2ae5113 tempest-VolumeQuotaClassesTest-1838313936 tempest-VolumeQuotaClassesTest-1838313936-project-admin] PATH_INFO: `/v3/projects/3b2d75c2fd9d4246a4f20e1a0229a9a7` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:04.823955 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944371e8-2882-426c-abe8-aa34b2ae5113 tempest-VolumeQuotaClassesTest-1838313936 tempest-VolumeQuotaClassesTest-1838313936-project-admin] RBAC: Authorizing `identity:delete_project(project_id=3b2d75c2fd9d4246a4f20e1a0229a9a7)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:04.827049 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944371e8-2882-426c-abe8-aa34b2ae5113 tempest-VolumeQuotaClassesTest-1838313936 tempest-VolumeQuotaClassesTest-1838313936-project-admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:04.905444 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-944371e8-2882-426c-abe8-aa34b2ae5113 tempest-VolumeQuotaClassesTest-1838313936 tempest-VolumeQuotaClassesTest-1838313936-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b2d75c2fd9d4246a4f20e1a0229a9a7', 'request_id': 'req-944371e8-2882-426c-abe8-aa34b2ae5113'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:04.916414 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-944371e8-2882-426c-abe8-aa34b2ae5113 tempest-VolumeQuotaClassesTest-1838313936 tempest-VolumeQuotaClassesTest-1838313936-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b2d75c2fd9d4246a4f20e1a0229a9a7', 'request_id': 'req-944371e8-2882-426c-abe8-aa34b2ae5113'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:04.924301 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-944371e8-2882-426c-abe8-aa34b2ae5113 tempest-VolumeQuotaClassesTest-1838313936 tempest-VolumeQuotaClassesTest-1838313936-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b2d75c2fd9d4246a4f20e1a0229a9a7', 'request_id': 'req-944371e8-2882-426c-abe8-aa34b2ae5113'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:04.929273 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:44:04.956529 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-944371e8-2882-426c-abe8-aa34b2ae5113 tempest-VolumeQuotaClassesTest-1838313936 tempest-VolumeQuotaClassesTest-1838313936-project-admin] The token cache is being invalidate because project 3b2d75c2fd9d4246a4f20e1a0229a9a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80154) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:44:04.957177 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-944371e8-2882-426c-abe8-aa34b2ae5113 tempest-VolumeQuotaClassesTest-1838313936 tempest-VolumeQuotaClassesTest-1838313936-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:04.958012 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-944371e8-2882-426c-abe8-aa34b2ae5113 tempest-VolumeQuotaClassesTest-1838313936 tempest-VolumeQuotaClassesTest-1838313936-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:04.960140 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1409/2819] 10.222.0.64 () {58 vars in 1233 bytes} [Tue Aug 2 10:44:04 2022] DELETE /identity/v3/projects/3b2d75c2fd9d4246a4f20e1a0229a9a7 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:44:05.204517 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a79d2ea-d8cf-42de-a58f-71f52a760858 tempest-VolumeQuotaClassesTest-1838313936 tempest-VolumeQuotaClassesTest-1838313936-project-admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:05.206720 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-8a79d2ea-d8cf-42de-a58f-71f52a760858 tempest-VolumeQuotaClassesTest-1838313936 tempest-VolumeQuotaClassesTest-1838313936-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:44:05.352771 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a79d2ea-d8cf-42de-a58f-71f52a760858 tempest-VolumeQuotaClassesTest-1838313936 tempest-VolumeQuotaClassesTest-1838313936-project-admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:05.409205 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944371e8-2882-426c-abe8-aa34b2ae5113 tempest-VolumeQuotaClassesTest-1838313936 tempest-VolumeQuotaClassesTest-1838313936-project-admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:05.410654 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-944371e8-2882-426c-abe8-aa34b2ae5113 tempest-VolumeQuotaClassesTest-1838313936 tempest-VolumeQuotaClassesTest-1838313936-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:44:05.473757 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88312f9b-4cc3-4980-8c88-025cfca4edda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:05.475090 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88312f9b-4cc3-4980-8c88-025cfca4edda admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:05.475602 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88312f9b-4cc3-4980-8c88-025cfca4edda admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:05.476113 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88312f9b-4cc3-4980-8c88-025cfca4edda admin admin] PATH_INFO: `/v3/projects/3775373d9a9f4f92bdb217d2017f7aec` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:05.477577 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88312f9b-4cc3-4980-8c88-025cfca4edda admin admin] RBAC: Authorizing `identity:delete_project(project_id=3775373d9a9f4f92bdb217d2017f7aec)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:05.479653 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88312f9b-4cc3-4980-8c88-025cfca4edda admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:05.543604 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944371e8-2882-426c-abe8-aa34b2ae5113 tempest-VolumeQuotaClassesTest-1838313936 tempest-VolumeQuotaClassesTest-1838313936-project-admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:05.570127 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-88312f9b-4cc3-4980-8c88-025cfca4edda admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3775373d9a9f4f92bdb217d2017f7aec', 'request_id': 'req-88312f9b-4cc3-4980-8c88-025cfca4edda'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:05.573981 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:44:05.581256 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-88312f9b-4cc3-4980-8c88-025cfca4edda admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3775373d9a9f4f92bdb217d2017f7aec', 'request_id': 'req-88312f9b-4cc3-4980-8c88-025cfca4edda'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:05.593292 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-88312f9b-4cc3-4980-8c88-025cfca4edda admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3775373d9a9f4f92bdb217d2017f7aec', 'request_id': 'req-88312f9b-4cc3-4980-8c88-025cfca4edda'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:05.615949 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-88312f9b-4cc3-4980-8c88-025cfca4edda admin admin] The token cache is being invalidate because project 3775373d9a9f4f92bdb217d2017f7aec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80153) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:44:05.616549 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-88312f9b-4cc3-4980-8c88-025cfca4edda admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:05.617490 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-88312f9b-4cc3-4980-8c88-025cfca4edda admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:05.621214 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1411/2820] 10.222.0.64 () {58 vars in 1233 bytes} [Tue Aug 2 10:44:05 2022] DELETE /identity/v3/projects/3775373d9a9f4f92bdb217d2017f7aec => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:44:05.625135 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88312f9b-4cc3-4980-8c88-025cfca4edda admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:05.627157 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-88312f9b-4cc3-4980-8c88-025cfca4edda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:44:05.653935 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935ef903-ab6b-4293-8ef9-bbadee423bf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:05.654722 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935ef903-ab6b-4293-8ef9-bbadee423bf9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:05.655082 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935ef903-ab6b-4293-8ef9-bbadee423bf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:05.655461 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935ef903-ab6b-4293-8ef9-bbadee423bf9 admin admin] PATH_INFO: `/v3/users/3342fcfdb30345faabdf616a332daacd` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:05.656907 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935ef903-ab6b-4293-8ef9-bbadee423bf9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3342fcfdb30345faabdf616a332daacd)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:05.659193 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935ef903-ab6b-4293-8ef9-bbadee423bf9 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:05.726355 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-935ef903-ab6b-4293-8ef9-bbadee423bf9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3342fcfdb30345faabdf616a332daacd'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:05.736931 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-935ef903-ab6b-4293-8ef9-bbadee423bf9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3342fcfdb30345faabdf616a332daacd'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:05.738140 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-935ef903-ab6b-4293-8ef9-bbadee423bf9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3342fcfdb30345faabdf616a332daacd'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:05.747770 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88312f9b-4cc3-4980-8c88-025cfca4edda admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:05.750229 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-935ef903-ab6b-4293-8ef9-bbadee423bf9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3342fcfdb30345faabdf616a332daacd'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:05.755185 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4ea237-9cdc-4cc7-9722-d236ec7d2aac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:05.755946 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4ea237-9cdc-4cc7-9722-d236ec7d2aac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:05.756466 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4ea237-9cdc-4cc7-9722-d236ec7d2aac admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:05.757084 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4ea237-9cdc-4cc7-9722-d236ec7d2aac admin admin] PATH_INFO: `/v3/users/034079393cb94d1780a03872b65ddd03` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:05.759257 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4ea237-9cdc-4cc7-9722-d236ec7d2aac admin admin] RBAC: Authorizing `identity:delete_user(user_id=034079393cb94d1780a03872b65ddd03)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:05.761573 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4ea237-9cdc-4cc7-9722-d236ec7d2aac admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:05.768972 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-935ef903-ab6b-4293-8ef9-bbadee423bf9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3342fcfdb30345faabdf616a332daacd'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:05.785542 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1410/2821] 10.222.0.64 () {58 vars in 1224 bytes} [Tue Aug 2 10:44:05 2022] DELETE /identity/v3/users/3342fcfdb30345faabdf616a332daacd => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:44:05.790998 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab9292c-4215-4646-93c3-82ccb26a8519 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:05.791293 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab9292c-4215-4646-93c3-82ccb26a8519 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:05.791813 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab9292c-4215-4646-93c3-82ccb26a8519 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:05.815664 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-7ab9292c-4215-4646-93c3-82ccb26a8519 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:44:05.815664 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-7ab9292c-4215-4646-93c3-82ccb26a8519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:44:05.844992 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:44:05.912339 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-9e4ea237-9cdc-4cc7-9722-d236ec7d2aac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '034079393cb94d1780a03872b65ddd03'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:05.925570 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-9e4ea237-9cdc-4cc7-9722-d236ec7d2aac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '034079393cb94d1780a03872b65ddd03'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:05.935381 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-9e4ea237-9cdc-4cc7-9722-d236ec7d2aac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '034079393cb94d1780a03872b65ddd03'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:05.936105 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-9e4ea237-9cdc-4cc7-9722-d236ec7d2aac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '034079393cb94d1780a03872b65ddd03'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:05.936876 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-9e4ea237-9cdc-4cc7-9722-d236ec7d2aac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '034079393cb94d1780a03872b65ddd03'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:05.951600 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1411/2822] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:44:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:44:05.962602 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab9292c-4215-4646-93c3-82ccb26a8519 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:05.964440 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-7ab9292c-4215-4646-93c3-82ccb26a8519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:44:05.968148 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1412/2823] 10.222.0.64 () {58 vars in 1224 bytes} [Tue Aug 2 10:44:05 2022] DELETE /identity/v3/users/034079393cb94d1780a03872b65ddd03 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:44:06.056341 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab9292c-4215-4646-93c3-82ccb26a8519 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:06.059637 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b404ae22-3ae2-4704-9681-2d9592ebf9e9 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:06.060543 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b404ae22-3ae2-4704-9681-2d9592ebf9e9 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:06.061045 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b404ae22-3ae2-4704-9681-2d9592ebf9e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:06.096510 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-b404ae22-3ae2-4704-9681-2d9592ebf9e9 None None] MFA Rules not processed for user `672442a55e9b47ccb8da313f8d3f4428`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:44:06.101796 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-b404ae22-3ae2-4704-9681-2d9592ebf9e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:44:06.176423 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e15f5a-593e-4ab3-adf0-9d4d9843fea6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:06.177494 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e15f5a-593e-4ab3-adf0-9d4d9843fea6 service neutron] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:06.178200 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e15f5a-593e-4ab3-adf0-9d4d9843fea6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:06.178618 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e15f5a-593e-4ab3-adf0-9d4d9843fea6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:06.235880 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1413/2824] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:44:06 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:44:06.276848 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e15f5a-593e-4ab3-adf0-9d4d9843fea6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:06.278516 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e15f5a-593e-4ab3-adf0-9d4d9843fea6 service neutron] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:06.293058 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1412/2825] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:44:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:44:06.324608 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b404ae22-3ae2-4704-9681-2d9592ebf9e9 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:06.328620 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-b404ae22-3ae2-4704-9681-2d9592ebf9e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:44:06.341619 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:44:06.361619 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b404ae22-3ae2-4704-9681-2d9592ebf9e9 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:06.375768 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a00083-815b-43cf-bb2d-92eb805bfa43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:06.376563 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a00083-815b-43cf-bb2d-92eb805bfa43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:06.376972 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a00083-815b-43cf-bb2d-92eb805bfa43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:06.377294 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a00083-815b-43cf-bb2d-92eb805bfa43 admin admin] PATH_INFO: `/v3/projects/d536c786011c480e87e1dd8c9e0e194f` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:06.378768 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a00083-815b-43cf-bb2d-92eb805bfa43 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d536c786011c480e87e1dd8c9e0e194f)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:06.382262 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a00083-815b-43cf-bb2d-92eb805bfa43 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:06.448640 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-40a00083-815b-43cf-bb2d-92eb805bfa43 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd536c786011c480e87e1dd8c9e0e194f', 'request_id': 'req-40a00083-815b-43cf-bb2d-92eb805bfa43'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:06.455339 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-40a00083-815b-43cf-bb2d-92eb805bfa43 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd536c786011c480e87e1dd8c9e0e194f', 'request_id': 'req-40a00083-815b-43cf-bb2d-92eb805bfa43'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:06.468059 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-40a00083-815b-43cf-bb2d-92eb805bfa43 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd536c786011c480e87e1dd8c9e0e194f', 'request_id': 'req-40a00083-815b-43cf-bb2d-92eb805bfa43'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:06.487902 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-40a00083-815b-43cf-bb2d-92eb805bfa43 admin admin] The token cache is being invalidate because project d536c786011c480e87e1dd8c9e0e194f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80153) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:44:06.489895 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-40a00083-815b-43cf-bb2d-92eb805bfa43 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:06.492369 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-40a00083-815b-43cf-bb2d-92eb805bfa43 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:06.495937 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1414/2826] 10.222.0.64 () {58 vars in 1233 bytes} [Tue Aug 2 10:44:06 2022] DELETE /identity/v3/projects/d536c786011c480e87e1dd8c9e0e194f => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:44:06.498285 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e15f5a-593e-4ab3-adf0-9d4d9843fea6 service neutron] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:06.512850 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e15f5a-593e-4ab3-adf0-9d4d9843fea6 service neutron] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:06.570522 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:44:06.620061 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3b45f9-57ab-4688-a60a-2c5e49474f37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:06.624643 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3b45f9-57ab-4688-a60a-2c5e49474f37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:06.624933 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3b45f9-57ab-4688-a60a-2c5e49474f37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:06.625216 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3b45f9-57ab-4688-a60a-2c5e49474f37 admin admin] PATH_INFO: `/v3/users/2e6b08f1ebf64329a1beb4b9bc8154bb` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:06.626997 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3b45f9-57ab-4688-a60a-2c5e49474f37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e6b08f1ebf64329a1beb4b9bc8154bb)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:06.628987 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3b45f9-57ab-4688-a60a-2c5e49474f37 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:06.765908 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a00083-815b-43cf-bb2d-92eb805bfa43 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:06.767385 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-40a00083-815b-43cf-bb2d-92eb805bfa43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:44:06.795747 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a00083-815b-43cf-bb2d-92eb805bfa43 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:06.806547 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-0c3b45f9-57ab-4688-a60a-2c5e49474f37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e6b08f1ebf64329a1beb4b9bc8154bb'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:06.807962 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b66e34-22f4-4c9b-90fb-4db580a920ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:06.808659 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b66e34-22f4-4c9b-90fb-4db580a920ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:06.808936 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b66e34-22f4-4c9b-90fb-4db580a920ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:06.809211 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b66e34-22f4-4c9b-90fb-4db580a920ef admin admin] PATH_INFO: `/v3/projects/801dc2387bda43cb905f9e8e6c5469c7` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:06.813465 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b66e34-22f4-4c9b-90fb-4db580a920ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=801dc2387bda43cb905f9e8e6c5469c7)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:06.818776 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b66e34-22f4-4c9b-90fb-4db580a920ef admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:06.824398 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-0c3b45f9-57ab-4688-a60a-2c5e49474f37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e6b08f1ebf64329a1beb4b9bc8154bb'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:06.829204 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-0c3b45f9-57ab-4688-a60a-2c5e49474f37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e6b08f1ebf64329a1beb4b9bc8154bb'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:06.841194 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-0c3b45f9-57ab-4688-a60a-2c5e49474f37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e6b08f1ebf64329a1beb4b9bc8154bb'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:06.867186 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-0c3b45f9-57ab-4688-a60a-2c5e49474f37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e6b08f1ebf64329a1beb4b9bc8154bb'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:06.875790 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1413/2827] 10.222.0.64 () {58 vars in 1224 bytes} [Tue Aug 2 10:44:06 2022] DELETE /identity/v3/users/2e6b08f1ebf64329a1beb4b9bc8154bb => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:44:06.943432 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-d5b66e34-22f4-4c9b-90fb-4db580a920ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '801dc2387bda43cb905f9e8e6c5469c7', 'request_id': 'req-d5b66e34-22f4-4c9b-90fb-4db580a920ef'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:06.966433 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-d5b66e34-22f4-4c9b-90fb-4db580a920ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '801dc2387bda43cb905f9e8e6c5469c7', 'request_id': 'req-d5b66e34-22f4-4c9b-90fb-4db580a920ef'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:06.989065 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-d5b66e34-22f4-4c9b-90fb-4db580a920ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '801dc2387bda43cb905f9e8e6c5469c7', 'request_id': 'req-d5b66e34-22f4-4c9b-90fb-4db580a920ef'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:07.012777 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-d5b66e34-22f4-4c9b-90fb-4db580a920ef admin admin] The token cache is being invalidate because project 801dc2387bda43cb905f9e8e6c5469c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80153) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:44:07.013155 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-d5b66e34-22f4-4c9b-90fb-4db580a920ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:07.014088 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-d5b66e34-22f4-4c9b-90fb-4db580a920ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:07.019139 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1415/2828] 10.222.0.64 () {58 vars in 1233 bytes} [Tue Aug 2 10:44:06 2022] DELETE /identity/v3/projects/801dc2387bda43cb905f9e8e6c5469c7 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:44:07.021228 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3b45f9-57ab-4688-a60a-2c5e49474f37 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:07.023474 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-0c3b45f9-57ab-4688-a60a-2c5e49474f37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:44:07.114425 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3b45f9-57ab-4688-a60a-2c5e49474f37 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:07.185435 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:44:07.207214 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239a3854-248f-4eb7-badf-31e5589de111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:07.208020 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239a3854-248f-4eb7-badf-31e5589de111 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:07.208413 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239a3854-248f-4eb7-badf-31e5589de111 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:07.208903 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239a3854-248f-4eb7-badf-31e5589de111 admin admin] PATH_INFO: `/v3/users/502828022f0647b4805168cd977dbfbb` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:07.210255 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239a3854-248f-4eb7-badf-31e5589de111 admin admin] RBAC: Authorizing `identity:delete_user(user_id=502828022f0647b4805168cd977dbfbb)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:07.213079 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239a3854-248f-4eb7-badf-31e5589de111 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:07.307353 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-239a3854-248f-4eb7-badf-31e5589de111 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '502828022f0647b4805168cd977dbfbb'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:07.319873 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b66e34-22f4-4c9b-90fb-4db580a920ef admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:07.321950 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-d5b66e34-22f4-4c9b-90fb-4db580a920ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:44:07.325314 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-239a3854-248f-4eb7-badf-31e5589de111 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '502828022f0647b4805168cd977dbfbb'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:07.326158 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-239a3854-248f-4eb7-badf-31e5589de111 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '502828022f0647b4805168cd977dbfbb'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:07.340508 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-239a3854-248f-4eb7-badf-31e5589de111 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '502828022f0647b4805168cd977dbfbb'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:07.341686 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b66e34-22f4-4c9b-90fb-4db580a920ef admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:07.347596 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9290af7-b166-4f9d-b3e3-02956a843adf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:07.348481 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9290af7-b166-4f9d-b3e3-02956a843adf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:07.348830 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9290af7-b166-4f9d-b3e3-02956a843adf admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:07.349179 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9290af7-b166-4f9d-b3e3-02956a843adf admin admin] PATH_INFO: `/v3/projects/ef9429f2fe2b46b38d016f43c99e26fb` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:07.351105 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9290af7-b166-4f9d-b3e3-02956a843adf admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef9429f2fe2b46b38d016f43c99e26fb)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:07.354669 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9290af7-b166-4f9d-b3e3-02956a843adf admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:07.356796 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-239a3854-248f-4eb7-badf-31e5589de111 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '502828022f0647b4805168cd977dbfbb'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:07.366779 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1414/2829] 10.222.0.64 () {58 vars in 1224 bytes} [Tue Aug 2 10:44:07 2022] DELETE /identity/v3/users/502828022f0647b4805168cd977dbfbb => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:44:07.419471 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-b9290af7-b166-4f9d-b3e3-02956a843adf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef9429f2fe2b46b38d016f43c99e26fb', 'request_id': 'req-b9290af7-b166-4f9d-b3e3-02956a843adf'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:07.426789 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-b9290af7-b166-4f9d-b3e3-02956a843adf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef9429f2fe2b46b38d016f43c99e26fb', 'request_id': 'req-b9290af7-b166-4f9d-b3e3-02956a843adf'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:07.434812 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-b9290af7-b166-4f9d-b3e3-02956a843adf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef9429f2fe2b46b38d016f43c99e26fb', 'request_id': 'req-b9290af7-b166-4f9d-b3e3-02956a843adf'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:07.437789 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=77,keystone:DELETE=20,keystone:INSERT=3 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:44:07.451564 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-b9290af7-b166-4f9d-b3e3-02956a843adf admin admin] The token cache is being invalidate because project ef9429f2fe2b46b38d016f43c99e26fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80153) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:44:07.452213 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-b9290af7-b166-4f9d-b3e3-02956a843adf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:07.453070 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-b9290af7-b166-4f9d-b3e3-02956a843adf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:07.455403 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1416/2830] 10.222.0.64 () {58 vars in 1233 bytes} [Tue Aug 2 10:44:07 2022] DELETE /identity/v3/projects/ef9429f2fe2b46b38d016f43c99e26fb => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:44:07.461628 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eef2ed-578c-40a8-82d7-4cd337931906 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:07.462592 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eef2ed-578c-40a8-82d7-4cd337931906 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:07.462891 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eef2ed-578c-40a8-82d7-4cd337931906 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:07.494799 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-16eef2ed-578c-40a8-82d7-4cd337931906 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:44:07.496205 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-16eef2ed-578c-40a8-82d7-4cd337931906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:44:07.615286 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1415/2831] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:44:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:44:07.624481 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0c4aa0-d725-4d2e-858d-9e62c98403fd None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:07.624801 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0c4aa0-d725-4d2e-858d-9e62c98403fd None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:07.625102 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0c4aa0-d725-4d2e-858d-9e62c98403fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:07.654656 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-8b0c4aa0-d725-4d2e-858d-9e62c98403fd None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:44:07.655988 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-8b0c4aa0-d725-4d2e-858d-9e62c98403fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:44:07.666716 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1417/2832] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:44:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:44:07.671189 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eef2ed-578c-40a8-82d7-4cd337931906 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:07.685569 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eef2ed-578c-40a8-82d7-4cd337931906 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:07.691368 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb703ec-9268-4387-a263-b91031881b6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:07.692002 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb703ec-9268-4387-a263-b91031881b6e admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:07.692317 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb703ec-9268-4387-a263-b91031881b6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:07.692601 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb703ec-9268-4387-a263-b91031881b6e admin admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:07.693320 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb703ec-9268-4387-a263-b91031881b6e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:44:07.693639 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb703ec-9268-4387-a263-b91031881b6e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:07.695753 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb703ec-9268-4387-a263-b91031881b6e admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:07.714732 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0c4aa0-d725-4d2e-858d-9e62c98403fd None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:07.720687 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1416/2833] 10.222.0.64 () {58 vars in 1153 bytes} [Tue Aug 2 10:44:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:44:07.724809 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0c4aa0-d725-4d2e-858d-9e62c98403fd None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:07.730491 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9fa601-0ffb-422f-8dd9-30008ef3d962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:07.731151 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9fa601-0ffb-422f-8dd9-30008ef3d962 admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:07.731427 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9fa601-0ffb-422f-8dd9-30008ef3d962 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:07.731712 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9fa601-0ffb-422f-8dd9-30008ef3d962 admin admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:07.732434 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9fa601-0ffb-422f-8dd9-30008ef3d962 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:07.734484 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9fa601-0ffb-422f-8dd9-30008ef3d962 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:07.769748 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb703ec-9268-4387-a263-b91031881b6e admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:07.771724 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-9cb703ec-9268-4387-a263-b91031881b6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:44:07.773761 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1418/2834] 10.222.0.64 () {60 vars in 1153 bytes} [Tue Aug 2 10:44:07 2022] POST /identity/v3/projects => generated 373 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:44:07.778338 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9fa601-0ffb-422f-8dd9-30008ef3d962 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:07.792641 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb703ec-9268-4387-a263-b91031881b6e admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:07.872107 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9b533e-cf3a-4c6d-ac68-c906d12d41a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:07.873554 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9b533e-cf3a-4c6d-ac68-c906d12d41a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:07.873904 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9b533e-cf3a-4c6d-ac68-c906d12d41a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:07.874246 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9b533e-cf3a-4c6d-ac68-c906d12d41a4 admin admin] PATH_INFO: `/v3/projects/c169bbd936994784873efe6ec7b15ca7` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:07.875898 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9b533e-cf3a-4c6d-ac68-c906d12d41a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c169bbd936994784873efe6ec7b15ca7)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:07.878704 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9b533e-cf3a-4c6d-ac68-c906d12d41a4 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:07.880451 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:44:07.883336 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9fa601-0ffb-422f-8dd9-30008ef3d962 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:07.895398 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8c6324-4c1e-436e-a4fb-b530d86f1b42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:07.896257 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8c6324-4c1e-436e-a4fb-b530d86f1b42 admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:07.896669 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8c6324-4c1e-436e-a4fb-b530d86f1b42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:07.897065 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8c6324-4c1e-436e-a4fb-b530d86f1b42 admin admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:07.898093 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8c6324-4c1e-436e-a4fb-b530d86f1b42 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:07.900276 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8c6324-4c1e-436e-a4fb-b530d86f1b42 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:07.924101 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1419/2835] 10.222.0.64 () {60 vars in 1144 bytes} [Tue Aug 2 10:44:07 2022] POST /identity/v3/users => generated 287 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:44:07.928317 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8c6324-4c1e-436e-a4fb-b530d86f1b42 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:07.939365 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8c6324-4c1e-436e-a4fb-b530d86f1b42 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:07.941705 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-8f9b533e-cf3a-4c6d-ac68-c906d12d41a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c169bbd936994784873efe6ec7b15ca7', 'request_id': 'req-8f9b533e-cf3a-4c6d-ac68-c906d12d41a4'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:07.954791 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-8f9b533e-cf3a-4c6d-ac68-c906d12d41a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c169bbd936994784873efe6ec7b15ca7', 'request_id': 'req-8f9b533e-cf3a-4c6d-ac68-c906d12d41a4'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:07.959308 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-8f9b533e-cf3a-4c6d-ac68-c906d12d41a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c169bbd936994784873efe6ec7b15ca7', 'request_id': 'req-8f9b533e-cf3a-4c6d-ac68-c906d12d41a4'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:07.976584 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-8f9b533e-cf3a-4c6d-ac68-c906d12d41a4 admin admin] The token cache is being invalidate because project c169bbd936994784873efe6ec7b15ca7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80154) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:44:07.979996 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-8f9b533e-cf3a-4c6d-ac68-c906d12d41a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:07.980825 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-8f9b533e-cf3a-4c6d-ac68-c906d12d41a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:07.985882 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1417/2836] 10.222.0.64 () {58 vars in 1233 bytes} [Tue Aug 2 10:44:07 2022] DELETE /identity/v3/projects/c169bbd936994784873efe6ec7b15ca7 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:44:07.987421 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e2a062-55a6-4367-9de7-2c9d732e1ce5 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:07.987750 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e2a062-55a6-4367-9de7-2c9d732e1ce5 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:07.988445 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e2a062-55a6-4367-9de7-2c9d732e1ce5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:08.018759 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-c3e2a062-55a6-4367-9de7-2c9d732e1ce5 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:44:08.020152 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-c3e2a062-55a6-4367-9de7-2c9d732e1ce5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:44:08.033052 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8495e855-07a6-420f-b6e2-f7067410c4fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:08.034098 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8495e855-07a6-420f-b6e2-f7067410c4fd admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:08.034453 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8495e855-07a6-420f-b6e2-f7067410c4fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:08.034798 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8495e855-07a6-420f-b6e2-f7067410c4fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:08.036166 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8495e855-07a6-420f-b6e2-f7067410c4fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:08.038038 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8495e855-07a6-420f-b6e2-f7067410c4fd admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:08.074202 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1420/2837] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:44:07 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:44:08.079127 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8495e855-07a6-420f-b6e2-f7067410c4fd admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:08.081873 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-8495e855-07a6-420f-b6e2-f7067410c4fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:44:08.097287 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8495e855-07a6-420f-b6e2-f7067410c4fd admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:08.102132 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1418/2838] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:44:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:44:08.103147 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be90201-7ad9-423b-9941-b7b77e758e2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:08.104349 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be90201-7ad9-423b-9941-b7b77e758e2b admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:08.104579 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be90201-7ad9-423b-9941-b7b77e758e2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:08.104860 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be90201-7ad9-423b-9941-b7b77e758e2b admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:08.105750 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be90201-7ad9-423b-9941-b7b77e758e2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:08.115892 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be90201-7ad9-423b-9941-b7b77e758e2b admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:08.116886 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437990f6-47f5-403f-9704-044230577f7a None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:08.117320 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437990f6-47f5-403f-9704-044230577f7a None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:08.117696 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437990f6-47f5-403f-9704-044230577f7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:08.134752 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1421/2839] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:44:08 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:44:08.135842 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be90201-7ad9-423b-9941-b7b77e758e2b admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:08.147515 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be90201-7ad9-423b-9941-b7b77e758e2b admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:08.152621 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-437990f6-47f5-403f-9704-044230577f7a None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:44:08.152867 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dbf958-b6d2-4e94-998e-81d3360ebf0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:08.154064 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-437990f6-47f5-403f-9704-044230577f7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:44:08.154259 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dbf958-b6d2-4e94-998e-81d3360ebf0a admin admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:08.154463 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dbf958-b6d2-4e94-998e-81d3360ebf0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:08.156355 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dbf958-b6d2-4e94-998e-81d3360ebf0a admin admin] PATH_INFO: `/v3/projects/b523a42badf7468f9d97925c5a84929a/users/5c3b254c5a834af1b1a89be588642204/roles/66c1074780954f8e87ed11a5c4e8c36a` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:08.178576 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1419/2840] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:44:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:44:08.180793 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437990f6-47f5-403f-9704-044230577f7a None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:08.189253 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dbf958-b6d2-4e94-998e-81d3360ebf0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b523a42badf7468f9d97925c5a84929a, user_id=5c3b254c5a834af1b1a89be588642204, role_id=66c1074780954f8e87ed11a5c4e8c36a)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:08.192935 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dbf958-b6d2-4e94-998e-81d3360ebf0a admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:08.202831 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437990f6-47f5-403f-9704-044230577f7a None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:08.210040 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1422/2841] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 10:44:08 2022] PUT /identity/v3/projects/b523a42badf7468f9d97925c5a84929a/users/5c3b254c5a834af1b1a89be588642204/roles/66c1074780954f8e87ed11a5c4e8c36a => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:44:08.221774 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0874c92-15a9-49c2-bc86-79bde37b608d None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:08.222068 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0874c92-15a9-49c2-bc86-79bde37b608d None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:08.222345 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0874c92-15a9-49c2-bc86-79bde37b608d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:08.259221 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-d0874c92-15a9-49c2-bc86-79bde37b608d None None] MFA Rules not processed for user `5c3b254c5a834af1b1a89be588642204`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:44:08.261029 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-d0874c92-15a9-49c2-bc86-79bde37b608d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:44:08.272397 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:DELETE=1,keystone:SELECT=94,keystone:INSERT=5 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:44:08.292995 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a298da-19d6-4ce5-b7c0-cea3175eebbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:08.293766 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a298da-19d6-4ce5-b7c0-cea3175eebbc admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:08.294102 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a298da-19d6-4ce5-b7c0-cea3175eebbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:08.294389 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a298da-19d6-4ce5-b7c0-cea3175eebbc admin admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:08.306372 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a298da-19d6-4ce5-b7c0-cea3175eebbc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:44:08.308053 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a298da-19d6-4ce5-b7c0-cea3175eebbc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:08.310017 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a298da-19d6-4ce5-b7c0-cea3175eebbc admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:08.333231 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1420/2842] 10.222.0.64 () {58 vars in 1153 bytes} [Tue Aug 2 10:44:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:44:08.338825 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a298da-19d6-4ce5-b7c0-cea3175eebbc admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:08.353398 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a298da-19d6-4ce5-b7c0-cea3175eebbc admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:08.360883 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb3afa0-db5b-4036-b9af-c1bc45c257c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:08.361536 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb3afa0-db5b-4036-b9af-c1bc45c257c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:08.361848 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb3afa0-db5b-4036-b9af-c1bc45c257c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:08.362102 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb3afa0-db5b-4036-b9af-c1bc45c257c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:08.362900 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb3afa0-db5b-4036-b9af-c1bc45c257c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:08.364994 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb3afa0-db5b-4036-b9af-c1bc45c257c2 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:08.386446 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:44:08.397558 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1423/2843] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:44:08 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:44:08.402101 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1421/2844] 10.222.0.64 () {60 vars in 1153 bytes} [Tue Aug 2 10:44:08 2022] POST /identity/v3/projects => generated 387 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:44:08.402942 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0874c92-15a9-49c2-bc86-79bde37b608d None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:08.405972 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb3afa0-db5b-4036-b9af-c1bc45c257c2 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:08.417889 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0874c92-15a9-49c2-bc86-79bde37b608d None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:08.486327 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb3afa0-db5b-4036-b9af-c1bc45c257c2 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:08.490328 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fbddae-ac69-4d18-8deb-cc83b587f60e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:08.491567 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fbddae-ac69-4d18-8deb-cc83b587f60e admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:08.491988 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fbddae-ac69-4d18-8deb-cc83b587f60e admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:08.492438 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fbddae-ac69-4d18-8deb-cc83b587f60e admin admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:08.494014 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fbddae-ac69-4d18-8deb-cc83b587f60e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:08.496319 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fbddae-ac69-4d18-8deb-cc83b587f60e admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:08.498324 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f814d1f2-df78-4c08-98c4-1f70b44c5e88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:08.499094 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f814d1f2-df78-4c08-98c4-1f70b44c5e88 admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:08.499752 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f814d1f2-df78-4c08-98c4-1f70b44c5e88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:08.500170 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f814d1f2-df78-4c08-98c4-1f70b44c5e88 admin admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:08.501071 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f814d1f2-df78-4c08-98c4-1f70b44c5e88 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:08.503694 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f814d1f2-df78-4c08-98c4-1f70b44c5e88 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:08.531039 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1422/2845] 10.222.0.64 () {60 vars in 1144 bytes} [Tue Aug 2 10:44:08 2022] POST /identity/v3/users => generated 294 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:44:08.532445 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1424/2846] 10.222.0.64 () {60 vars in 1153 bytes} [Tue Aug 2 10:44:08 2022] POST /identity/v3/projects => generated 373 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:44:08.536508 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f814d1f2-df78-4c08-98c4-1f70b44c5e88 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:08.538183 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fbddae-ac69-4d18-8deb-cc83b587f60e admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:08.548220 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f814d1f2-df78-4c08-98c4-1f70b44c5e88 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:08.622023 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6eb93f-d94a-4dfa-8800-cfc2f98d4a3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:08.622756 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6eb93f-d94a-4dfa-8800-cfc2f98d4a3c admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:08.623144 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6eb93f-d94a-4dfa-8800-cfc2f98d4a3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:08.623520 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6eb93f-d94a-4dfa-8800-cfc2f98d4a3c admin admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:08.624366 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6eb93f-d94a-4dfa-8800-cfc2f98d4a3c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:08.626391 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6eb93f-d94a-4dfa-8800-cfc2f98d4a3c admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:08.640292 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fbddae-ac69-4d18-8deb-cc83b587f60e admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:08.646918 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceba735e-a6c5-43ed-b507-f4ca39183a58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:08.647552 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceba735e-a6c5-43ed-b507-f4ca39183a58 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:08.647916 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceba735e-a6c5-43ed-b507-f4ca39183a58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:08.648120 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceba735e-a6c5-43ed-b507-f4ca39183a58 admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:08.648841 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceba735e-a6c5-43ed-b507-f4ca39183a58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:08.651435 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceba735e-a6c5-43ed-b507-f4ca39183a58 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:08.656467 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1423/2847] 10.222.0.64 () {60 vars in 1144 bytes} [Tue Aug 2 10:44:08 2022] POST /identity/v3/users => generated 293 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:44:08.660099 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6eb93f-d94a-4dfa-8800-cfc2f98d4a3c admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:08.670316 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1425/2848] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:44:08 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:44:08.671582 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6eb93f-d94a-4dfa-8800-cfc2f98d4a3c admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:08.674027 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceba735e-a6c5-43ed-b507-f4ca39183a58 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:08.680035 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec4cce4-5b17-4147-8ae9-fd6489c35006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:08.680661 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec4cce4-5b17-4147-8ae9-fd6489c35006 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:08.680994 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec4cce4-5b17-4147-8ae9-fd6489c35006 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:08.681771 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec4cce4-5b17-4147-8ae9-fd6489c35006 admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:08.681977 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec4cce4-5b17-4147-8ae9-fd6489c35006 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:08.683525 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec4cce4-5b17-4147-8ae9-fd6489c35006 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:08.687693 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceba735e-a6c5-43ed-b507-f4ca39183a58 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:08.694277 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedf4cd2-b63c-4535-807e-c940c310a54d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:08.695365 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedf4cd2-b63c-4535-807e-c940c310a54d admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:08.695522 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1424/2849] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:44:08 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:44:08.695672 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedf4cd2-b63c-4535-807e-c940c310a54d admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:08.700921 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedf4cd2-b63c-4535-807e-c940c310a54d admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:08.700921 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedf4cd2-b63c-4535-807e-c940c310a54d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:08.700921 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedf4cd2-b63c-4535-807e-c940c310a54d admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:08.702600 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec4cce4-5b17-4147-8ae9-fd6489c35006 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:08.714099 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec4cce4-5b17-4147-8ae9-fd6489c35006 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:08.715470 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1426/2850] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:44:08 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:44:08.719477 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedf4cd2-b63c-4535-807e-c940c310a54d admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:08.722495 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49435115-8253-4fd6-b77b-dc3f865d1d1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:08.723235 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49435115-8253-4fd6-b77b-dc3f865d1d1d admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:08.723761 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49435115-8253-4fd6-b77b-dc3f865d1d1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:08.724110 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49435115-8253-4fd6-b77b-dc3f865d1d1d admin admin] PATH_INFO: `/v3/projects/6b29f85aa40d4f2f8019886b981e658c/users/426d773bb4b3413bbaacbf63ba081c6d/roles/1000cf318fa14915b8091599eaead5de` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:08.733158 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedf4cd2-b63c-4535-807e-c940c310a54d admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:08.747221 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a863166-02f9-4ec3-882f-5f72229b96b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:08.748371 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a863166-02f9-4ec3-882f-5f72229b96b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:08.749005 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a863166-02f9-4ec3-882f-5f72229b96b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:08.749445 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a863166-02f9-4ec3-882f-5f72229b96b0 admin admin] PATH_INFO: `/v3/projects/253d03a91af94bbba3aa02a5a2fe4d4c/users/863c508dd44a4d108acbf2b3f3831d51/roles/66c1074780954f8e87ed11a5c4e8c36a` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:08.749597 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49435115-8253-4fd6-b77b-dc3f865d1d1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b29f85aa40d4f2f8019886b981e658c, user_id=426d773bb4b3413bbaacbf63ba081c6d, role_id=1000cf318fa14915b8091599eaead5de)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:08.752442 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49435115-8253-4fd6-b77b-dc3f865d1d1d admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:08.768152 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:44:08.772056 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1425/2851] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 10:44:08 2022] PUT /identity/v3/projects/6b29f85aa40d4f2f8019886b981e658c/users/426d773bb4b3413bbaacbf63ba081c6d/roles/1000cf318fa14915b8091599eaead5de => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:44:08.777256 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49435115-8253-4fd6-b77b-dc3f865d1d1d admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:08.777641 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a863166-02f9-4ec3-882f-5f72229b96b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=253d03a91af94bbba3aa02a5a2fe4d4c, user_id=863c508dd44a4d108acbf2b3f3831d51, role_id=66c1074780954f8e87ed11a5c4e8c36a)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:08.782042 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a863166-02f9-4ec3-882f-5f72229b96b0 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:08.813625 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1427/2852] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 10:44:08 2022] PUT /identity/v3/projects/253d03a91af94bbba3aa02a5a2fe4d4c/users/863c508dd44a4d108acbf2b3f3831d51/roles/66c1074780954f8e87ed11a5c4e8c36a => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:44:08.820958 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d22278-6e91-4e61-b06e-abf1a27e19ac None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:08.828341 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d22278-6e91-4e61-b06e-abf1a27e19ac None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:08.828341 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d22278-6e91-4e61-b06e-abf1a27e19ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:08.834429 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:44:08.866253 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49435115-8253-4fd6-b77b-dc3f865d1d1d admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:08.875744 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-21d22278-6e91-4e61-b06e-abf1a27e19ac None None] MFA Rules not processed for user `863c508dd44a4d108acbf2b3f3831d51`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:44:08.877241 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-21d22278-6e91-4e61-b06e-abf1a27e19ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:44:08.938809 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3641237-59ab-4a6b-aaa2-f19c903cc64d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:08.939406 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3641237-59ab-4a6b-aaa2-f19c903cc64d admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:08.939945 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3641237-59ab-4a6b-aaa2-f19c903cc64d admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:08.940228 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3641237-59ab-4a6b-aaa2-f19c903cc64d admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:08.941103 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3641237-59ab-4a6b-aaa2-f19c903cc64d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:08.943018 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3641237-59ab-4a6b-aaa2-f19c903cc64d admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:08.956694 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1426/2853] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:44:08 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:44:08.958236 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1428/2854] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:44:08 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:44:08.960137 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3641237-59ab-4a6b-aaa2-f19c903cc64d admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:08.964146 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d22278-6e91-4e61-b06e-abf1a27e19ac None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:08.978721 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3641237-59ab-4a6b-aaa2-f19c903cc64d admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:08.979950 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d22278-6e91-4e61-b06e-abf1a27e19ac None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:08.983597 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec85ff81-4e20-4601-84db-1c4935682cfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:08.984355 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec85ff81-4e20-4601-84db-1c4935682cfd admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:08.984645 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec85ff81-4e20-4601-84db-1c4935682cfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:08.984926 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec85ff81-4e20-4601-84db-1c4935682cfd admin admin] PATH_INFO: `/v3/projects/6b29f85aa40d4f2f8019886b981e658c/users/426d773bb4b3413bbaacbf63ba081c6d/roles/1000cf318fa14915b8091599eaead5de` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:08.986343 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25bf043-86cc-4389-9fcf-1c2f1510e70b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:08.986926 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec85ff81-4e20-4601-84db-1c4935682cfd admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b29f85aa40d4f2f8019886b981e658c, user_id=426d773bb4b3413bbaacbf63ba081c6d, role_id=1000cf318fa14915b8091599eaead5de)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:08.987093 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25bf043-86cc-4389-9fcf-1c2f1510e70b admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:08.987447 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25bf043-86cc-4389-9fcf-1c2f1510e70b admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:08.987783 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25bf043-86cc-4389-9fcf-1c2f1510e70b admin admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:08.988558 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25bf043-86cc-4389-9fcf-1c2f1510e70b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:08.989613 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec85ff81-4e20-4601-84db-1c4935682cfd admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:08.990595 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25bf043-86cc-4389-9fcf-1c2f1510e70b admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:09.000880 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1427/2855] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 10:44:08 2022] PUT /identity/v3/projects/6b29f85aa40d4f2f8019886b981e658c/users/426d773bb4b3413bbaacbf63ba081c6d/roles/1000cf318fa14915b8091599eaead5de => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:44:09.016266 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022d3ecc-a3dd-4539-b672-6f62e400d5b7 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:09.018661 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022d3ecc-a3dd-4539-b672-6f62e400d5b7 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:09.018956 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022d3ecc-a3dd-4539-b672-6f62e400d5b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:09.023889 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1429/2856] 10.222.0.64 () {60 vars in 1153 bytes} [Tue Aug 2 10:44:08 2022] POST /identity/v3/projects => generated 387 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:44:09.028495 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25bf043-86cc-4389-9fcf-1c2f1510e70b admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:09.050176 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-022d3ecc-a3dd-4539-b672-6f62e400d5b7 None None] MFA Rules not processed for user `426d773bb4b3413bbaacbf63ba081c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:44:09.051459 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-022d3ecc-a3dd-4539-b672-6f62e400d5b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:44:09.129046 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25bf043-86cc-4389-9fcf-1c2f1510e70b admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:09.143469 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f1f973-398b-4cff-aa49-3a0795e3a750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:09.144102 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f1f973-398b-4cff-aa49-3a0795e3a750 admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:09.144420 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f1f973-398b-4cff-aa49-3a0795e3a750 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:09.144700 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f1f973-398b-4cff-aa49-3a0795e3a750 admin admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:09.145377 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f1f973-398b-4cff-aa49-3a0795e3a750 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:09.147333 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f1f973-398b-4cff-aa49-3a0795e3a750 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:09.167883 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1428/2857] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:44:09 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:44:09.171703 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1430/2858] 10.222.0.64 () {60 vars in 1144 bytes} [Tue Aug 2 10:44:09 2022] POST /identity/v3/users => generated 300 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:44:09.173250 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022d3ecc-a3dd-4539-b672-6f62e400d5b7 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:09.182557 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f1f973-398b-4cff-aa49-3a0795e3a750 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:09.184654 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-c7f1f973-398b-4cff-aa49-3a0795e3a750 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:44:09.187787 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022d3ecc-a3dd-4539-b672-6f62e400d5b7 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:09.193404 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29641de4-82ba-4001-8c76-0b8c2ebdf2a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:09.194226 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29641de4-82ba-4001-8c76-0b8c2ebdf2a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:09.194507 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29641de4-82ba-4001-8c76-0b8c2ebdf2a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:09.194827 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29641de4-82ba-4001-8c76-0b8c2ebdf2a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:09.195571 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29641de4-82ba-4001-8c76-0b8c2ebdf2a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:09.197964 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29641de4-82ba-4001-8c76-0b8c2ebdf2a4 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:09.209301 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1429/2859] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:44:09 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:44:09.213450 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29641de4-82ba-4001-8c76-0b8c2ebdf2a4 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:09.226300 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29641de4-82ba-4001-8c76-0b8c2ebdf2a4 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:09.232652 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30c99a2-2f91-4513-84f5-10da9bfd8e2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:09.233343 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30c99a2-2f91-4513-84f5-10da9bfd8e2b admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:09.233629 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30c99a2-2f91-4513-84f5-10da9bfd8e2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:09.233908 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30c99a2-2f91-4513-84f5-10da9bfd8e2b admin admin] PATH_INFO: `/v3/projects/4701bd10f839439dac310232c5d6aa92/users/099f26d0b176442a8bc6d11e2d48dc1c/roles/1000cf318fa14915b8091599eaead5de` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:09.246604 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:44:09.251449 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f1f973-398b-4cff-aa49-3a0795e3a750 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:09.251577 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30c99a2-2f91-4513-84f5-10da9bfd8e2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=4701bd10f839439dac310232c5d6aa92, user_id=099f26d0b176442a8bc6d11e2d48dc1c, role_id=1000cf318fa14915b8091599eaead5de)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:09.254215 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30c99a2-2f91-4513-84f5-10da9bfd8e2b admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:09.256353 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbf99fe-4e8a-479d-9298-a77c8563acf9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d7de62b9ee4039bdc79627ac7fb499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:09.257276 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbf99fe-4e8a-479d-9298-a77c8563acf9 service cinder] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:09.257887 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbf99fe-4e8a-479d-9298-a77c8563acf9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:09.257887 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbf99fe-4e8a-479d-9298-a77c8563acf9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:09.269916 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1430/2860] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 10:44:09 2022] PUT /identity/v3/projects/4701bd10f839439dac310232c5d6aa92/users/099f26d0b176442a8bc6d11e2d48dc1c/roles/1000cf318fa14915b8091599eaead5de => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:44:09.273939 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30c99a2-2f91-4513-84f5-10da9bfd8e2b admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:09.299960 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:44:09.334443 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbf99fe-4e8a-479d-9298-a77c8563acf9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:09.336279 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbf99fe-4e8a-479d-9298-a77c8563acf9 service cinder] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:09.347736 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1431/2861] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:44:09 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:44:09.347886 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30c99a2-2f91-4513-84f5-10da9bfd8e2b admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:09.352871 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6219617-9929-40ca-963e-894ef2dc4fc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:09.353556 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6219617-9929-40ca-963e-894ef2dc4fc6 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:09.353846 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6219617-9929-40ca-963e-894ef2dc4fc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:09.354118 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6219617-9929-40ca-963e-894ef2dc4fc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:09.354831 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6219617-9929-40ca-963e-894ef2dc4fc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:09.356609 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6219617-9929-40ca-963e-894ef2dc4fc6 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:09.374912 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1431/2862] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:44:09 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:44:09.377409 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbf99fe-4e8a-479d-9298-a77c8563acf9 service cinder] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:09.389580 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbf99fe-4e8a-479d-9298-a77c8563acf9 service cinder] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:09.395440 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca96caeb-2d6a-4c10-ab92-1faaa434155c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:09.396259 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca96caeb-2d6a-4c10-ab92-1faaa434155c admin admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:09.396698 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca96caeb-2d6a-4c10-ab92-1faaa434155c admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:09.397560 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca96caeb-2d6a-4c10-ab92-1faaa434155c admin admin] PATH_INFO: `/v3/projects/4701bd10f839439dac310232c5d6aa92/users/099f26d0b176442a8bc6d11e2d48dc1c/roles/1000cf318fa14915b8091599eaead5de` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:09.400386 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca96caeb-2d6a-4c10-ab92-1faaa434155c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4701bd10f839439dac310232c5d6aa92, user_id=099f26d0b176442a8bc6d11e2d48dc1c, role_id=1000cf318fa14915b8091599eaead5de)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:09.403790 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca96caeb-2d6a-4c10-ab92-1faaa434155c admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:09.425537 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1432/2863] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 10:44:09 2022] PUT /identity/v3/projects/4701bd10f839439dac310232c5d6aa92/users/099f26d0b176442a8bc6d11e2d48dc1c/roles/1000cf318fa14915b8091599eaead5de => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:44:09.428433 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf16c88-86b7-4c5f-8372-4f966ea15cca None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:09.429150 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf16c88-86b7-4c5f-8372-4f966ea15cca None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:09.429150 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf16c88-86b7-4c5f-8372-4f966ea15cca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:09.461270 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-3cf16c88-86b7-4c5f-8372-4f966ea15cca None None] MFA Rules not processed for user `099f26d0b176442a8bc6d11e2d48dc1c`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:44:09.462705 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-3cf16c88-86b7-4c5f-8372-4f966ea15cca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:44:09.542523 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1432/2864] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:44:09 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:44:09.558920 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca96caeb-2d6a-4c10-ab92-1faaa434155c admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:09.561682 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-ca96caeb-2d6a-4c10-ab92-1faaa434155c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:44:09.650596 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca96caeb-2d6a-4c10-ab92-1faaa434155c admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:09.655195 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe41d4f-5756-4f27-9bb7-f268eff3ab7a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d7de62b9ee4039bdc79627ac7fb499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:09.656704 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe41d4f-5756-4f27-9bb7-f268eff3ab7a service cinder] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:09.657076 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe41d4f-5756-4f27-9bb7-f268eff3ab7a service cinder] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:09.657261 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe41d4f-5756-4f27-9bb7-f268eff3ab7a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:09.669033 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe41d4f-5756-4f27-9bb7-f268eff3ab7a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:09.670729 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe41d4f-5756-4f27-9bb7-f268eff3ab7a service cinder] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:09.683208 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1433/2865] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:44:09 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:44:10.318334 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf16c88-86b7-4c5f-8372-4f966ea15cca None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:10.393623 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf16c88-86b7-4c5f-8372-4f966ea15cca None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:10.400379 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb0857c-30dc-4bc6-a92c-1215e023aeb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:10.401103 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb0857c-30dc-4bc6-a92c-1215e023aeb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:10.401407 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb0857c-30dc-4bc6-a92c-1215e023aeb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:10.401685 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb0857c-30dc-4bc6-a92c-1215e023aeb0 admin admin] PATH_INFO: `/v3/users/863c508dd44a4d108acbf2b3f3831d51` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:10.405822 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb0857c-30dc-4bc6-a92c-1215e023aeb0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=863c508dd44a4d108acbf2b3f3831d51)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:10.407887 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb0857c-30dc-4bc6-a92c-1215e023aeb0 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:10.458135 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:44:10.461180 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-8cb0857c-30dc-4bc6-a92c-1215e023aeb0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '863c508dd44a4d108acbf2b3f3831d51'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:10.482569 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-8cb0857c-30dc-4bc6-a92c-1215e023aeb0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '863c508dd44a4d108acbf2b3f3831d51'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:10.483438 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-8cb0857c-30dc-4bc6-a92c-1215e023aeb0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '863c508dd44a4d108acbf2b3f3831d51'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:10.492088 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-8cb0857c-30dc-4bc6-a92c-1215e023aeb0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '863c508dd44a4d108acbf2b3f3831d51'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:10.506075 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-8cb0857c-30dc-4bc6-a92c-1215e023aeb0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '863c508dd44a4d108acbf2b3f3831d51'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:10.513576 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1433/2866] 10.222.0.64 () {58 vars in 1224 bytes} [Tue Aug 2 10:44:10 2022] DELETE /identity/v3/users/863c508dd44a4d108acbf2b3f3831d51 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:44:10.517772 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8010f6-f8c1-43a4-a171-7837bf01c52d None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:10.518095 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8010f6-f8c1-43a4-a171-7837bf01c52d None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:10.518340 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8010f6-f8c1-43a4-a171-7837bf01c52d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:10.541026 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-8e8010f6-f8c1-43a4-a171-7837bf01c52d None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:44:10.545342 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-8e8010f6-f8c1-43a4-a171-7837bf01c52d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:44:10.611199 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1434/2867] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:44:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:44:10.620959 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb0857c-30dc-4bc6-a92c-1215e023aeb0 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:10.622216 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-8cb0857c-30dc-4bc6-a92c-1215e023aeb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:44:10.681676 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb0857c-30dc-4bc6-a92c-1215e023aeb0 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:10.685320 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b24c15-a0a6-4636-9b24-8ebfe555439e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:10.685937 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b24c15-a0a6-4636-9b24-8ebfe555439e service neutron] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:10.686221 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b24c15-a0a6-4636-9b24-8ebfe555439e service neutron] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:10.686497 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b24c15-a0a6-4636-9b24-8ebfe555439e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:10.695461 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b24c15-a0a6-4636-9b24-8ebfe555439e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:10.696965 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b24c15-a0a6-4636-9b24-8ebfe555439e service neutron] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:10.706355 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1434/2868] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:44:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:44:10.991355 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8010f6-f8c1-43a4-a171-7837bf01c52d None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:10.992798 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-8e8010f6-f8c1-43a4-a171-7837bf01c52d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:44:11.006792 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8010f6-f8c1-43a4-a171-7837bf01c52d None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:11.016567 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd2fc94-9c10-4b85-ab6f-f2582b95e27f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:11.017347 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd2fc94-9c10-4b85-ab6f-f2582b95e27f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:11.017694 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd2fc94-9c10-4b85-ab6f-f2582b95e27f admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:11.017973 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd2fc94-9c10-4b85-ab6f-f2582b95e27f admin admin] PATH_INFO: `/v3/projects/253d03a91af94bbba3aa02a5a2fe4d4c` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:11.019143 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd2fc94-9c10-4b85-ab6f-f2582b95e27f admin admin] RBAC: Authorizing `identity:delete_project(project_id=253d03a91af94bbba3aa02a5a2fe4d4c)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:11.021176 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd2fc94-9c10-4b85-ab6f-f2582b95e27f admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:11.052526 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:44:11.075321 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-bdd2fc94-9c10-4b85-ab6f-f2582b95e27f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '253d03a91af94bbba3aa02a5a2fe4d4c', 'request_id': 'req-bdd2fc94-9c10-4b85-ab6f-f2582b95e27f'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:11.080870 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-bdd2fc94-9c10-4b85-ab6f-f2582b95e27f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '253d03a91af94bbba3aa02a5a2fe4d4c', 'request_id': 'req-bdd2fc94-9c10-4b85-ab6f-f2582b95e27f'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:11.087255 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-bdd2fc94-9c10-4b85-ab6f-f2582b95e27f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '253d03a91af94bbba3aa02a5a2fe4d4c', 'request_id': 'req-bdd2fc94-9c10-4b85-ab6f-f2582b95e27f'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:11.099843 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-bdd2fc94-9c10-4b85-ab6f-f2582b95e27f admin admin] The token cache is being invalidate because project 253d03a91af94bbba3aa02a5a2fe4d4c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80153) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:44:11.100046 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-bdd2fc94-9c10-4b85-ab6f-f2582b95e27f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:11.100794 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-bdd2fc94-9c10-4b85-ab6f-f2582b95e27f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:11.102286 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1435/2869] 10.222.0.64 () {58 vars in 1233 bytes} [Tue Aug 2 10:44:10 2022] DELETE /identity/v3/projects/253d03a91af94bbba3aa02a5a2fe4d4c => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:44:11.106985 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b24c15-a0a6-4636-9b24-8ebfe555439e service neutron] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:11.117847 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b24c15-a0a6-4636-9b24-8ebfe555439e service neutron] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:11.176151 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fb4d25-b504-436a-a076-1f8813959723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:11.176823 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fb4d25-b504-436a-a076-1f8813959723 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:11.177108 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fb4d25-b504-436a-a076-1f8813959723 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:11.177383 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fb4d25-b504-436a-a076-1f8813959723 admin admin] PATH_INFO: `/v3/users/099f26d0b176442a8bc6d11e2d48dc1c` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:11.178807 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fb4d25-b504-436a-a076-1f8813959723 admin admin] RBAC: Authorizing `identity:delete_user(user_id=099f26d0b176442a8bc6d11e2d48dc1c)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:11.181422 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fb4d25-b504-436a-a076-1f8813959723 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:11.232209 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-19fb4d25-b504-436a-a076-1f8813959723 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '099f26d0b176442a8bc6d11e2d48dc1c'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:11.239223 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-19fb4d25-b504-436a-a076-1f8813959723 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '099f26d0b176442a8bc6d11e2d48dc1c'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:11.239897 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-19fb4d25-b504-436a-a076-1f8813959723 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '099f26d0b176442a8bc6d11e2d48dc1c'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:11.248393 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-19fb4d25-b504-436a-a076-1f8813959723 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '099f26d0b176442a8bc6d11e2d48dc1c'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:11.250349 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:44:11.269818 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-19fb4d25-b504-436a-a076-1f8813959723 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '099f26d0b176442a8bc6d11e2d48dc1c'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:11.278487 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1435/2870] 10.222.0.64 () {58 vars in 1224 bytes} [Tue Aug 2 10:44:11 2022] DELETE /identity/v3/users/099f26d0b176442a8bc6d11e2d48dc1c => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:44:11.564889 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd2fc94-9c10-4b85-ab6f-f2582b95e27f admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:11.566563 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-bdd2fc94-9c10-4b85-ab6f-f2582b95e27f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:44:11.586255 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd2fc94-9c10-4b85-ab6f-f2582b95e27f admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:11.642866 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d3e295-a3c2-433b-a904-d5177dc2d3a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:11.643849 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d3e295-a3c2-433b-a904-d5177dc2d3a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:11.644375 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d3e295-a3c2-433b-a904-d5177dc2d3a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:11.644887 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d3e295-a3c2-433b-a904-d5177dc2d3a3 admin admin] PATH_INFO: `/v3/projects/4701bd10f839439dac310232c5d6aa92` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:11.646912 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d3e295-a3c2-433b-a904-d5177dc2d3a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4701bd10f839439dac310232c5d6aa92)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:11.650685 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d3e295-a3c2-433b-a904-d5177dc2d3a3 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:11.701513 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-d3d3e295-a3c2-433b-a904-d5177dc2d3a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4701bd10f839439dac310232c5d6aa92', 'request_id': 'req-d3d3e295-a3c2-433b-a904-d5177dc2d3a3'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:11.707050 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-d3d3e295-a3c2-433b-a904-d5177dc2d3a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4701bd10f839439dac310232c5d6aa92', 'request_id': 'req-d3d3e295-a3c2-433b-a904-d5177dc2d3a3'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:11.713413 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-d3d3e295-a3c2-433b-a904-d5177dc2d3a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4701bd10f839439dac310232c5d6aa92', 'request_id': 'req-d3d3e295-a3c2-433b-a904-d5177dc2d3a3'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:11.729135 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-d3d3e295-a3c2-433b-a904-d5177dc2d3a3 admin admin] The token cache is being invalidate because project 4701bd10f839439dac310232c5d6aa92 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80153) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:44:11.729440 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-d3d3e295-a3c2-433b-a904-d5177dc2d3a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:11.730118 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-d3d3e295-a3c2-433b-a904-d5177dc2d3a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:11.731722 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1436/2871] 10.222.0.64 () {58 vars in 1233 bytes} [Tue Aug 2 10:44:11 2022] DELETE /identity/v3/projects/4701bd10f839439dac310232c5d6aa92 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:44:11.737329 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d96891-166a-4a21-97c0-7e6bcd20dcee None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:11.737807 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d96891-166a-4a21-97c0-7e6bcd20dcee None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:11.737935 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d96891-166a-4a21-97c0-7e6bcd20dcee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:11.758031 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-97d96891-166a-4a21-97c0-7e6bcd20dcee None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:44:11.759290 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-97d96891-166a-4a21-97c0-7e6bcd20dcee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:44:11.822896 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1436/2872] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:44:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:44:11.830206 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dc117d-11bd-40a7-bd4e-0772f55816d1 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:11.830466 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dc117d-11bd-40a7-bd4e-0772f55816d1 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:11.830744 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dc117d-11bd-40a7-bd4e-0772f55816d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:11.850535 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-60dc117d-11bd-40a7-bd4e-0772f55816d1 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:44:11.851738 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-60dc117d-11bd-40a7-bd4e-0772f55816d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:44:11.862214 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1437/2873] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:44:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:44:11.866390 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d96891-166a-4a21-97c0-7e6bcd20dcee None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:11.877970 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d96891-166a-4a21-97c0-7e6bcd20dcee None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:11.882708 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be3762b-e606-4362-afd2-123b1be28c0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:11.883419 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be3762b-e606-4362-afd2-123b1be28c0e admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:11.886730 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be3762b-e606-4362-afd2-123b1be28c0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:11.886730 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be3762b-e606-4362-afd2-123b1be28c0e admin admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:11.886730 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be3762b-e606-4362-afd2-123b1be28c0e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:44:11.886730 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be3762b-e606-4362-afd2-123b1be28c0e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:11.886730 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be3762b-e606-4362-afd2-123b1be28c0e admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:11.898924 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1437/2874] 10.222.0.64 () {58 vars in 1153 bytes} [Tue Aug 2 10:44:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:44:11.903310 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dc117d-11bd-40a7-bd4e-0772f55816d1 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:11.912720 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dc117d-11bd-40a7-bd4e-0772f55816d1 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:11.917959 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44f44cc-7f15-4cfa-8ed3-2a0c67c36b23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:11.918562 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44f44cc-7f15-4cfa-8ed3-2a0c67c36b23 admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:11.918849 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44f44cc-7f15-4cfa-8ed3-2a0c67c36b23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:11.919118 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44f44cc-7f15-4cfa-8ed3-2a0c67c36b23 admin admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:11.919787 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44f44cc-7f15-4cfa-8ed3-2a0c67c36b23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:11.921706 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44f44cc-7f15-4cfa-8ed3-2a0c67c36b23 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:11.954503 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1438/2875] 10.222.0.64 () {60 vars in 1153 bytes} [Tue Aug 2 10:44:11 2022] POST /identity/v3/projects => generated 385 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:44:11.958722 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be3762b-e606-4362-afd2-123b1be28c0e admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:11.969520 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be3762b-e606-4362-afd2-123b1be28c0e admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:12.050256 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f153db-31c2-4eaa-b1c2-a244fb784703 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:12.050926 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f153db-31c2-4eaa-b1c2-a244fb784703 admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:12.051308 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f153db-31c2-4eaa-b1c2-a244fb784703 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:12.051620 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f153db-31c2-4eaa-b1c2-a244fb784703 admin admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:12.052395 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f153db-31c2-4eaa-b1c2-a244fb784703 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:12.054670 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f153db-31c2-4eaa-b1c2-a244fb784703 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:12.078641 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1438/2876] 10.222.0.64 () {60 vars in 1144 bytes} [Tue Aug 2 10:44:11 2022] POST /identity/v3/users => generated 293 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:44:12.083003 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44f44cc-7f15-4cfa-8ed3-2a0c67c36b23 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:12.094421 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44f44cc-7f15-4cfa-8ed3-2a0c67c36b23 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:12.099250 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859e0a91-f2ef-4ef7-92b3-68786be5efec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:12.099866 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859e0a91-f2ef-4ef7-92b3-68786be5efec admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:12.100171 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859e0a91-f2ef-4ef7-92b3-68786be5efec admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:12.100467 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859e0a91-f2ef-4ef7-92b3-68786be5efec admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:12.101174 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859e0a91-f2ef-4ef7-92b3-68786be5efec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:12.103510 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859e0a91-f2ef-4ef7-92b3-68786be5efec admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:12.113920 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1439/2877] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:44:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:44:12.118056 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f153db-31c2-4eaa-b1c2-a244fb784703 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:12.128162 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f153db-31c2-4eaa-b1c2-a244fb784703 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:12.132832 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa87368-81ed-4220-b8f0-e04a0c2d739f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:12.133509 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa87368-81ed-4220-b8f0-e04a0c2d739f admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:12.133796 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa87368-81ed-4220-b8f0-e04a0c2d739f admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:12.134067 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa87368-81ed-4220-b8f0-e04a0c2d739f admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:12.134758 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa87368-81ed-4220-b8f0-e04a0c2d739f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:12.136362 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa87368-81ed-4220-b8f0-e04a0c2d739f admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:12.146806 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1439/2878] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:44:12 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:44:12.151512 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859e0a91-f2ef-4ef7-92b3-68786be5efec admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:12.161525 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859e0a91-f2ef-4ef7-92b3-68786be5efec admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:12.166827 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a0245b-ab25-4332-a5f9-805d74c5bfa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:12.167478 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a0245b-ab25-4332-a5f9-805d74c5bfa5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:12.167794 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a0245b-ab25-4332-a5f9-805d74c5bfa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:12.168072 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a0245b-ab25-4332-a5f9-805d74c5bfa5 admin admin] PATH_INFO: `/v3/projects/3efda2dff4784b37b047acc487385e3e/users/627617e0c3c6410f88b1bdb55856e7bf/roles/66c1074780954f8e87ed11a5c4e8c36a` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:12.170339 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:44:12.185302 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a0245b-ab25-4332-a5f9-805d74c5bfa5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3efda2dff4784b37b047acc487385e3e, user_id=627617e0c3c6410f88b1bdb55856e7bf, role_id=66c1074780954f8e87ed11a5c4e8c36a)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:12.187942 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a0245b-ab25-4332-a5f9-805d74c5bfa5 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:12.200446 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1440/2879] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 10:44:12 2022] PUT /identity/v3/projects/3efda2dff4784b37b047acc487385e3e/users/627617e0c3c6410f88b1bdb55856e7bf/roles/66c1074780954f8e87ed11a5c4e8c36a => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:44:12.214882 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c900622b-d2e9-45e1-8b5b-4b8801932917 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:12.215269 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c900622b-d2e9-45e1-8b5b-4b8801932917 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:12.215584 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c900622b-d2e9-45e1-8b5b-4b8801932917 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:12.243580 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-c900622b-d2e9-45e1-8b5b-4b8801932917 None None] MFA Rules not processed for user `627617e0c3c6410f88b1bdb55856e7bf`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:44:12.244834 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-c900622b-d2e9-45e1-8b5b-4b8801932917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:44:12.288942 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:44:12.332079 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1440/2880] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:44:12 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:44:12.336983 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a0245b-ab25-4332-a5f9-805d74c5bfa5 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:12.407510 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a0245b-ab25-4332-a5f9-805d74c5bfa5 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:12.412487 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b372b6d-6481-4f26-81e5-6f46f8dd5144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:12.413129 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b372b6d-6481-4f26-81e5-6f46f8dd5144 admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:12.413438 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b372b6d-6481-4f26-81e5-6f46f8dd5144 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:12.413721 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b372b6d-6481-4f26-81e5-6f46f8dd5144 admin admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:12.414522 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b372b6d-6481-4f26-81e5-6f46f8dd5144 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:12.417160 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b372b6d-6481-4f26-81e5-6f46f8dd5144 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:12.448900 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1441/2881] 10.222.0.64 () {60 vars in 1153 bytes} [Tue Aug 2 10:44:12 2022] POST /identity/v3/projects => generated 383 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:44:12.450674 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c900622b-d2e9-45e1-8b5b-4b8801932917 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:12.462546 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c900622b-d2e9-45e1-8b5b-4b8801932917 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:12.520321 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020ccde4-a00b-4cd2-a1c1-baab91431f01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:12.520949 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020ccde4-a00b-4cd2-a1c1-baab91431f01 admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:12.521234 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020ccde4-a00b-4cd2-a1c1-baab91431f01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:12.521506 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020ccde4-a00b-4cd2-a1c1-baab91431f01 admin admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:12.522172 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020ccde4-a00b-4cd2-a1c1-baab91431f01 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:12.524076 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020ccde4-a00b-4cd2-a1c1-baab91431f01 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:12.546611 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1441/2882] 10.222.0.64 () {60 vars in 1144 bytes} [Tue Aug 2 10:44:12 2022] POST /identity/v3/users => generated 298 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:44:12.550828 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b372b6d-6481-4f26-81e5-6f46f8dd5144 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:12.562299 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b372b6d-6481-4f26-81e5-6f46f8dd5144 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:12.567832 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a30b3e-3e85-4265-879f-88902865da22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:12.568506 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a30b3e-3e85-4265-879f-88902865da22 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:12.568978 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a30b3e-3e85-4265-879f-88902865da22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:12.569372 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a30b3e-3e85-4265-879f-88902865da22 admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:12.569966 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a30b3e-3e85-4265-879f-88902865da22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:12.573007 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a30b3e-3e85-4265-879f-88902865da22 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:12.589098 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1442/2883] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:44:12 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:44:12.592190 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020ccde4-a00b-4cd2-a1c1-baab91431f01 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:12.602235 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020ccde4-a00b-4cd2-a1c1-baab91431f01 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:12.607503 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c65601-2551-4bb3-9779-e939fabf51b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:12.608164 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c65601-2551-4bb3-9779-e939fabf51b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:12.608469 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c65601-2551-4bb3-9779-e939fabf51b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:12.608771 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c65601-2551-4bb3-9779-e939fabf51b3 admin admin] PATH_INFO: `/v3/projects/761322b96375471aa8595ee7c876e767/users/9ac459e8d09b4c1ba9c7b5289feb234d/roles/1000cf318fa14915b8091599eaead5de` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:12.624167 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c65601-2551-4bb3-9779-e939fabf51b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=761322b96375471aa8595ee7c876e767, user_id=9ac459e8d09b4c1ba9c7b5289feb234d, role_id=1000cf318fa14915b8091599eaead5de)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:12.627085 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c65601-2551-4bb3-9779-e939fabf51b3 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:12.642318 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1442/2884] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 10:44:12 2022] PUT /identity/v3/projects/761322b96375471aa8595ee7c876e767/users/9ac459e8d09b4c1ba9c7b5289feb234d/roles/1000cf318fa14915b8091599eaead5de => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:44:12.646319 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a30b3e-3e85-4265-879f-88902865da22 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:12.657036 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a30b3e-3e85-4265-879f-88902865da22 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:12.714167 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401021a5-7875-4d4f-8586-3280b4d91f36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:12.714702 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401021a5-7875-4d4f-8586-3280b4d91f36 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:12.714985 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401021a5-7875-4d4f-8586-3280b4d91f36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:12.715261 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401021a5-7875-4d4f-8586-3280b4d91f36 admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:12.715947 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401021a5-7875-4d4f-8586-3280b4d91f36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:12.717584 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401021a5-7875-4d4f-8586-3280b4d91f36 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:12.727185 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1443/2885] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:44:12 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:44:12.731784 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c65601-2551-4bb3-9779-e939fabf51b3 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:12.744644 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c65601-2551-4bb3-9779-e939fabf51b3 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:12.749915 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e21084b-8443-41a4-88a4-145d9c4b0dd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:12.750602 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e21084b-8443-41a4-88a4-145d9c4b0dd7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:12.750890 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e21084b-8443-41a4-88a4-145d9c4b0dd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:12.751164 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e21084b-8443-41a4-88a4-145d9c4b0dd7 admin admin] PATH_INFO: `/v3/projects/761322b96375471aa8595ee7c876e767/users/9ac459e8d09b4c1ba9c7b5289feb234d/roles/1000cf318fa14915b8091599eaead5de` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:12.753166 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e21084b-8443-41a4-88a4-145d9c4b0dd7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=761322b96375471aa8595ee7c876e767, user_id=9ac459e8d09b4c1ba9c7b5289feb234d, role_id=1000cf318fa14915b8091599eaead5de)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:12.755882 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e21084b-8443-41a4-88a4-145d9c4b0dd7 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:12.767227 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1443/2886] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 10:44:12 2022] PUT /identity/v3/projects/761322b96375471aa8595ee7c876e767/users/9ac459e8d09b4c1ba9c7b5289feb234d/roles/1000cf318fa14915b8091599eaead5de => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:44:12.778182 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28cfe45-1258-44cd-8049-4e7a6947611e None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:12.778475 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28cfe45-1258-44cd-8049-4e7a6947611e None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:12.778782 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28cfe45-1258-44cd-8049-4e7a6947611e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:12.809567 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-a28cfe45-1258-44cd-8049-4e7a6947611e None None] MFA Rules not processed for user `9ac459e8d09b4c1ba9c7b5289feb234d`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:44:12.810827 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-a28cfe45-1258-44cd-8049-4e7a6947611e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:44:12.852459 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:44:12.882152 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1444/2887] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:44:12 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:44:12.893447 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e21084b-8443-41a4-88a4-145d9c4b0dd7 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:12.895219 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-2e21084b-8443-41a4-88a4-145d9c4b0dd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:44:12.961744 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e21084b-8443-41a4-88a4-145d9c4b0dd7 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:12.965988 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a160ab2c-a8de-4ecd-a6ba-818eaedcd1cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d7de62b9ee4039bdc79627ac7fb499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:12.966543 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a160ab2c-a8de-4ecd-a6ba-818eaedcd1cd service cinder] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:12.966826 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a160ab2c-a8de-4ecd-a6ba-818eaedcd1cd service cinder] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:12.967103 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a160ab2c-a8de-4ecd-a6ba-818eaedcd1cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:13.012014 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:44:13.026649 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a160ab2c-a8de-4ecd-a6ba-818eaedcd1cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:13.028098 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a160ab2c-a8de-4ecd-a6ba-818eaedcd1cd service cinder] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:13.037154 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1444/2888] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:44:12 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:44:13.074011 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28cfe45-1258-44cd-8049-4e7a6947611e None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:13.075600 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-a28cfe45-1258-44cd-8049-4e7a6947611e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:44:13.153805 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28cfe45-1258-44cd-8049-4e7a6947611e None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:13.158720 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1568613f-fbbc-4e03-9c99-0f5e5412da15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:13.159363 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1568613f-fbbc-4e03-9c99-0f5e5412da15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:13.159655 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1568613f-fbbc-4e03-9c99-0f5e5412da15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:13.159933 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1568613f-fbbc-4e03-9c99-0f5e5412da15 admin admin] PATH_INFO: `/v3/users/5c3b254c5a834af1b1a89be588642204` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:13.161207 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1568613f-fbbc-4e03-9c99-0f5e5412da15 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c3b254c5a834af1b1a89be588642204)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:13.163564 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1568613f-fbbc-4e03-9c99-0f5e5412da15 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:13.224963 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-1568613f-fbbc-4e03-9c99-0f5e5412da15 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c3b254c5a834af1b1a89be588642204'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:13.232893 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-1568613f-fbbc-4e03-9c99-0f5e5412da15 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c3b254c5a834af1b1a89be588642204'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:13.241325 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-1568613f-fbbc-4e03-9c99-0f5e5412da15 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c3b254c5a834af1b1a89be588642204'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:13.241943 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-1568613f-fbbc-4e03-9c99-0f5e5412da15 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c3b254c5a834af1b1a89be588642204'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:13.242550 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-1568613f-fbbc-4e03-9c99-0f5e5412da15 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c3b254c5a834af1b1a89be588642204'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:13.261737 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1445/2889] 10.222.0.64 () {58 vars in 1224 bytes} [Tue Aug 2 10:44:13 2022] DELETE /identity/v3/users/5c3b254c5a834af1b1a89be588642204 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:44:13.266396 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe2dc4c-2278-4310-bf1f-c618a81f83ac None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:13.266933 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe2dc4c-2278-4310-bf1f-c618a81f83ac None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:13.267511 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe2dc4c-2278-4310-bf1f-c618a81f83ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:13.294602 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-7fe2dc4c-2278-4310-bf1f-c618a81f83ac None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:44:13.295831 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-7fe2dc4c-2278-4310-bf1f-c618a81f83ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:44:13.372738 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1445/2890] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:44:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:44:13.386804 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1568613f-fbbc-4e03-9c99-0f5e5412da15 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:13.388838 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-1568613f-fbbc-4e03-9c99-0f5e5412da15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:44:13.478059 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1568613f-fbbc-4e03-9c99-0f5e5412da15 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:13.481735 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ce0f65-92e4-4b9d-babe-901491bd1cb1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:13.482431 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ce0f65-92e4-4b9d-babe-901491bd1cb1 service neutron] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:13.482637 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ce0f65-92e4-4b9d-babe-901491bd1cb1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:13.482912 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ce0f65-92e4-4b9d-babe-901491bd1cb1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:13.492464 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ce0f65-92e4-4b9d-babe-901491bd1cb1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:13.493965 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ce0f65-92e4-4b9d-babe-901491bd1cb1 service neutron] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:13.499040 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:44:13.504509 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1446/2891] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:44:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:44:13.778064 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe2dc4c-2278-4310-bf1f-c618a81f83ac None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:13.779779 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-7fe2dc4c-2278-4310-bf1f-c618a81f83ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:44:13.794525 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe2dc4c-2278-4310-bf1f-c618a81f83ac None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:13.799400 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9485bef-3e43-45eb-8815-2788599329d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:13.800064 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9485bef-3e43-45eb-8815-2788599329d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:13.800390 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9485bef-3e43-45eb-8815-2788599329d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:13.800657 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9485bef-3e43-45eb-8815-2788599329d3 admin admin] PATH_INFO: `/v3/projects/b523a42badf7468f9d97925c5a84929a` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:13.802569 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9485bef-3e43-45eb-8815-2788599329d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b523a42badf7468f9d97925c5a84929a)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:13.804800 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9485bef-3e43-45eb-8815-2788599329d3 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:13.860865 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-e9485bef-3e43-45eb-8815-2788599329d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b523a42badf7468f9d97925c5a84929a', 'request_id': 'req-e9485bef-3e43-45eb-8815-2788599329d3'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:13.866370 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-e9485bef-3e43-45eb-8815-2788599329d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b523a42badf7468f9d97925c5a84929a', 'request_id': 'req-e9485bef-3e43-45eb-8815-2788599329d3'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:13.872537 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-e9485bef-3e43-45eb-8815-2788599329d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b523a42badf7468f9d97925c5a84929a', 'request_id': 'req-e9485bef-3e43-45eb-8815-2788599329d3'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:13.885267 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-e9485bef-3e43-45eb-8815-2788599329d3 admin admin] The token cache is being invalidate because project b523a42badf7468f9d97925c5a84929a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80154) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:44:13.885562 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-e9485bef-3e43-45eb-8815-2788599329d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:13.886206 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-e9485bef-3e43-45eb-8815-2788599329d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:13.887731 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1446/2892] 10.222.0.64 () {58 vars in 1233 bytes} [Tue Aug 2 10:44:13 2022] DELETE /identity/v3/projects/b523a42badf7468f9d97925c5a84929a => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:44:13.891689 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ce0f65-92e4-4b9d-babe-901491bd1cb1 service neutron] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:13.903672 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ce0f65-92e4-4b9d-babe-901491bd1cb1 service neutron] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:13.962311 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd8d34d-cc35-4a36-8c9b-489aff2eb294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:13.962996 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd8d34d-cc35-4a36-8c9b-489aff2eb294 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:13.963306 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd8d34d-cc35-4a36-8c9b-489aff2eb294 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:13.963589 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd8d34d-cc35-4a36-8c9b-489aff2eb294 admin admin] PATH_INFO: `/v3/users/426d773bb4b3413bbaacbf63ba081c6d` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:13.964792 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd8d34d-cc35-4a36-8c9b-489aff2eb294 admin admin] RBAC: Authorizing `identity:delete_user(user_id=426d773bb4b3413bbaacbf63ba081c6d)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:13.966730 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd8d34d-cc35-4a36-8c9b-489aff2eb294 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:14.014437 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-ffd8d34d-cc35-4a36-8c9b-489aff2eb294 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '426d773bb4b3413bbaacbf63ba081c6d'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:14.022286 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-ffd8d34d-cc35-4a36-8c9b-489aff2eb294 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '426d773bb4b3413bbaacbf63ba081c6d'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:14.029438 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-ffd8d34d-cc35-4a36-8c9b-489aff2eb294 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '426d773bb4b3413bbaacbf63ba081c6d'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:14.030029 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-ffd8d34d-cc35-4a36-8c9b-489aff2eb294 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '426d773bb4b3413bbaacbf63ba081c6d'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:14.030636 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-ffd8d34d-cc35-4a36-8c9b-489aff2eb294 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '426d773bb4b3413bbaacbf63ba081c6d'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:14.048683 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1447/2893] 10.222.0.64 () {58 vars in 1224 bytes} [Tue Aug 2 10:44:13 2022] DELETE /identity/v3/users/426d773bb4b3413bbaacbf63ba081c6d => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:44:14.597330 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9485bef-3e43-45eb-8815-2788599329d3 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:14.598929 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-e9485bef-3e43-45eb-8815-2788599329d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:44:14.613333 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9485bef-3e43-45eb-8815-2788599329d3 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:14.683442 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd024468-7f77-426d-ae74-d9dd6cb14e0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:14.684215 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd024468-7f77-426d-ae74-d9dd6cb14e0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:14.684543 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd024468-7f77-426d-ae74-d9dd6cb14e0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:14.684820 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd024468-7f77-426d-ae74-d9dd6cb14e0a admin admin] PATH_INFO: `/v3/projects/6b29f85aa40d4f2f8019886b981e658c` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:14.686066 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd024468-7f77-426d-ae74-d9dd6cb14e0a admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b29f85aa40d4f2f8019886b981e658c)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:14.687966 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd024468-7f77-426d-ae74-d9dd6cb14e0a admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:14.699659 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:44:14.736030 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-fd024468-7f77-426d-ae74-d9dd6cb14e0a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b29f85aa40d4f2f8019886b981e658c', 'request_id': 'req-fd024468-7f77-426d-ae74-d9dd6cb14e0a'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:14.742083 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-fd024468-7f77-426d-ae74-d9dd6cb14e0a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b29f85aa40d4f2f8019886b981e658c', 'request_id': 'req-fd024468-7f77-426d-ae74-d9dd6cb14e0a'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:14.748823 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-fd024468-7f77-426d-ae74-d9dd6cb14e0a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b29f85aa40d4f2f8019886b981e658c', 'request_id': 'req-fd024468-7f77-426d-ae74-d9dd6cb14e0a'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:14.762389 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-fd024468-7f77-426d-ae74-d9dd6cb14e0a admin admin] The token cache is being invalidate because project 6b29f85aa40d4f2f8019886b981e658c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80154) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:44:14.762682 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-fd024468-7f77-426d-ae74-d9dd6cb14e0a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:14.763325 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-fd024468-7f77-426d-ae74-d9dd6cb14e0a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:14.764904 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1447/2894] 10.222.0.64 () {58 vars in 1233 bytes} [Tue Aug 2 10:44:14 2022] DELETE /identity/v3/projects/6b29f85aa40d4f2f8019886b981e658c => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:44:14.769704 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae7cb3f-4ffc-4c75-a977-3187a2474841 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:14.769997 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae7cb3f-4ffc-4c75-a977-3187a2474841 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:14.770279 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae7cb3f-4ffc-4c75-a977-3187a2474841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:14.791750 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-dae7cb3f-4ffc-4c75-a977-3187a2474841 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:44:14.793193 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-dae7cb3f-4ffc-4c75-a977-3187a2474841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:44:14.858740 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1448/2895] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:44:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:44:14.866868 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296670f2-12c0-46f1-a47e-8d94ddffc46e None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:14.867390 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296670f2-12c0-46f1-a47e-8d94ddffc46e None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:14.867390 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296670f2-12c0-46f1-a47e-8d94ddffc46e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:14.887994 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-296670f2-12c0-46f1-a47e-8d94ddffc46e None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:44:14.889307 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-296670f2-12c0-46f1-a47e-8d94ddffc46e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:44:14.897366 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1448/2896] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:44:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:44:14.902326 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae7cb3f-4ffc-4c75-a977-3187a2474841 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:14.911980 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae7cb3f-4ffc-4c75-a977-3187a2474841 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:14.916636 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c38aa48-8b5b-456e-8ae0-71aa405af916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:14.917246 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c38aa48-8b5b-456e-8ae0-71aa405af916 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:14.917530 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c38aa48-8b5b-456e-8ae0-71aa405af916 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:14.917807 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c38aa48-8b5b-456e-8ae0-71aa405af916 admin admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:14.918502 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c38aa48-8b5b-456e-8ae0-71aa405af916 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:44:14.918812 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c38aa48-8b5b-456e-8ae0-71aa405af916 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:14.920440 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c38aa48-8b5b-456e-8ae0-71aa405af916 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:14.932540 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1449/2897] 10.222.0.64 () {58 vars in 1153 bytes} [Tue Aug 2 10:44:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:44:14.938674 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296670f2-12c0-46f1-a47e-8d94ddffc46e None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:14.949568 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296670f2-12c0-46f1-a47e-8d94ddffc46e None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:14.954280 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea67f8c6-2bea-4384-92f6-52498556b83f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:14.954891 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea67f8c6-2bea-4384-92f6-52498556b83f admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:14.955177 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea67f8c6-2bea-4384-92f6-52498556b83f admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:14.955454 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea67f8c6-2bea-4384-92f6-52498556b83f admin admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:14.956134 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea67f8c6-2bea-4384-92f6-52498556b83f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:14.958071 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea67f8c6-2bea-4384-92f6-52498556b83f admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:14.985778 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1449/2898] 10.222.0.64 () {60 vars in 1153 bytes} [Tue Aug 2 10:44:14 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:44:14.990188 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c38aa48-8b5b-456e-8ae0-71aa405af916 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:15.002984 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c38aa48-8b5b-456e-8ae0-71aa405af916 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:15.007454 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:44:15.065983 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ac2e45-e07a-4e1f-9404-63082d4a52cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:15.066651 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ac2e45-e07a-4e1f-9404-63082d4a52cc admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:15.066939 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ac2e45-e07a-4e1f-9404-63082d4a52cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:15.067246 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ac2e45-e07a-4e1f-9404-63082d4a52cc admin admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:15.068004 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ac2e45-e07a-4e1f-9404-63082d4a52cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:15.070165 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ac2e45-e07a-4e1f-9404-63082d4a52cc admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:15.093356 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1450/2899] 10.222.0.64 () {60 vars in 1144 bytes} [Tue Aug 2 10:44:14 2022] POST /identity/v3/users => generated 287 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:44:15.096768 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea67f8c6-2bea-4384-92f6-52498556b83f admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:15.107524 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea67f8c6-2bea-4384-92f6-52498556b83f admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:15.112326 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2d1735-eaf1-4a0f-ad53-ac5563059d4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:15.112941 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2d1735-eaf1-4a0f-ad53-ac5563059d4a admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:15.113226 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2d1735-eaf1-4a0f-ad53-ac5563059d4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:15.113502 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2d1735-eaf1-4a0f-ad53-ac5563059d4a admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:15.114203 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2d1735-eaf1-4a0f-ad53-ac5563059d4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:15.116018 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2d1735-eaf1-4a0f-ad53-ac5563059d4a admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:15.129920 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1450/2900] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:44:15 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:44:15.134055 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ac2e45-e07a-4e1f-9404-63082d4a52cc admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:15.144427 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ac2e45-e07a-4e1f-9404-63082d4a52cc admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:15.149219 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4afebd-0577-4d9c-95b5-3c0002c65c00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:15.150473 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4afebd-0577-4d9c-95b5-3c0002c65c00 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:15.150755 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4afebd-0577-4d9c-95b5-3c0002c65c00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:15.151028 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4afebd-0577-4d9c-95b5-3c0002c65c00 admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:15.151725 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4afebd-0577-4d9c-95b5-3c0002c65c00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:15.153401 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4afebd-0577-4d9c-95b5-3c0002c65c00 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:15.163969 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1451/2901] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:44:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:44:15.168691 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2d1735-eaf1-4a0f-ad53-ac5563059d4a admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:15.182476 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2d1735-eaf1-4a0f-ad53-ac5563059d4a admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:15.187276 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7be50b-a861-407c-884f-addb3dc7ac84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:15.187947 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7be50b-a861-407c-884f-addb3dc7ac84 admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:15.188233 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7be50b-a861-407c-884f-addb3dc7ac84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:15.188546 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7be50b-a861-407c-884f-addb3dc7ac84 admin admin] PATH_INFO: `/v3/projects/8169d7fe3f0440468b0c9ceda7dfb2de/users/d01e846239e74e368dbf91901b24a267/roles/66c1074780954f8e87ed11a5c4e8c36a` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:15.203713 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7be50b-a861-407c-884f-addb3dc7ac84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8169d7fe3f0440468b0c9ceda7dfb2de, user_id=d01e846239e74e368dbf91901b24a267, role_id=66c1074780954f8e87ed11a5c4e8c36a)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:15.206496 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7be50b-a861-407c-884f-addb3dc7ac84 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:15.220158 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1451/2902] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 10:44:15 2022] PUT /identity/v3/projects/8169d7fe3f0440468b0c9ceda7dfb2de/users/d01e846239e74e368dbf91901b24a267/roles/66c1074780954f8e87ed11a5c4e8c36a => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:44:15.231536 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9248d3a8-d313-49fb-b78e-2e85904e8970 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:15.231865 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9248d3a8-d313-49fb-b78e-2e85904e8970 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:15.232181 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9248d3a8-d313-49fb-b78e-2e85904e8970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:15.260527 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-9248d3a8-d313-49fb-b78e-2e85904e8970 None None] MFA Rules not processed for user `d01e846239e74e368dbf91901b24a267`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:44:15.261756 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-9248d3a8-d313-49fb-b78e-2e85904e8970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:44:15.330963 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1452/2903] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:44:15 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:44:15.343008 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7be50b-a861-407c-884f-addb3dc7ac84 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:15.344793 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-fd7be50b-a861-407c-884f-addb3dc7ac84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:44:15.404817 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7be50b-a861-407c-884f-addb3dc7ac84 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:15.408450 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e7f690-3fe8-4a2e-ab88-77228b340ddf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d7de62b9ee4039bdc79627ac7fb499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:15.409104 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e7f690-3fe8-4a2e-ab88-77228b340ddf service cinder] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:15.409389 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e7f690-3fe8-4a2e-ab88-77228b340ddf service cinder] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:15.409682 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e7f690-3fe8-4a2e-ab88-77228b340ddf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:15.418509 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e7f690-3fe8-4a2e-ab88-77228b340ddf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:15.419938 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e7f690-3fe8-4a2e-ab88-77228b340ddf service cinder] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:15.428849 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1452/2904] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:44:15 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:44:15.455666 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9248d3a8-d313-49fb-b78e-2e85904e8970 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:15.525800 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9248d3a8-d313-49fb-b78e-2e85904e8970 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:15.530945 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767d6d17-4da9-40b7-af9b-9795e8a91234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:15.531663 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767d6d17-4da9-40b7-af9b-9795e8a91234 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:15.531950 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767d6d17-4da9-40b7-af9b-9795e8a91234 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:15.532218 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767d6d17-4da9-40b7-af9b-9795e8a91234 admin admin] PATH_INFO: `/v3/users/d01e846239e74e368dbf91901b24a267` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:15.533447 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767d6d17-4da9-40b7-af9b-9795e8a91234 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d01e846239e74e368dbf91901b24a267)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:15.535561 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767d6d17-4da9-40b7-af9b-9795e8a91234 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:15.565413 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:44:15.601379 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-767d6d17-4da9-40b7-af9b-9795e8a91234 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd01e846239e74e368dbf91901b24a267'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:15.609621 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-767d6d17-4da9-40b7-af9b-9795e8a91234 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd01e846239e74e368dbf91901b24a267'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:15.616866 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-767d6d17-4da9-40b7-af9b-9795e8a91234 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd01e846239e74e368dbf91901b24a267'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:15.617582 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-767d6d17-4da9-40b7-af9b-9795e8a91234 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd01e846239e74e368dbf91901b24a267'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:15.618874 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-767d6d17-4da9-40b7-af9b-9795e8a91234 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd01e846239e74e368dbf91901b24a267'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:15.638133 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1453/2905] 10.222.0.64 () {58 vars in 1224 bytes} [Tue Aug 2 10:44:15 2022] DELETE /identity/v3/users/d01e846239e74e368dbf91901b24a267 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:44:15.642669 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58b1db8-8db8-4475-8832-2c8009faee0d None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:15.643002 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58b1db8-8db8-4475-8832-2c8009faee0d None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:15.643296 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58b1db8-8db8-4475-8832-2c8009faee0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:15.663139 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-d58b1db8-8db8-4475-8832-2c8009faee0d None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:44:15.664448 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-d58b1db8-8db8-4475-8832-2c8009faee0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:44:15.713065 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:44:15.764554 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1453/2906] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:44:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:44:15.774760 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767d6d17-4da9-40b7-af9b-9795e8a91234 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:15.776224 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-767d6d17-4da9-40b7-af9b-9795e8a91234 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:44:15.845841 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767d6d17-4da9-40b7-af9b-9795e8a91234 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:15.850513 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549483e6-b4ed-49c9-b215-e719be86545b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:15.851393 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549483e6-b4ed-49c9-b215-e719be86545b service neutron] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:15.851683 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549483e6-b4ed-49c9-b215-e719be86545b service neutron] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:15.851970 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549483e6-b4ed-49c9-b215-e719be86545b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:15.865337 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549483e6-b4ed-49c9-b215-e719be86545b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:15.869632 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549483e6-b4ed-49c9-b215-e719be86545b service neutron] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:15.891691 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1454/2907] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:44:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:44:16.187516 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58b1db8-8db8-4475-8832-2c8009faee0d None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:16.189034 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-d58b1db8-8db8-4475-8832-2c8009faee0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:44:16.203455 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58b1db8-8db8-4475-8832-2c8009faee0d None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:16.208436 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67152695-4521-4541-a280-ebe56202ddb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:16.209758 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67152695-4521-4541-a280-ebe56202ddb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:16.210242 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67152695-4521-4541-a280-ebe56202ddb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:16.210534 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67152695-4521-4541-a280-ebe56202ddb4 admin admin] PATH_INFO: `/v3/projects/8169d7fe3f0440468b0c9ceda7dfb2de` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:16.211749 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67152695-4521-4541-a280-ebe56202ddb4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8169d7fe3f0440468b0c9ceda7dfb2de)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:16.214337 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67152695-4521-4541-a280-ebe56202ddb4 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:16.263470 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-67152695-4521-4541-a280-ebe56202ddb4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8169d7fe3f0440468b0c9ceda7dfb2de', 'request_id': 'req-67152695-4521-4541-a280-ebe56202ddb4'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:16.268676 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-67152695-4521-4541-a280-ebe56202ddb4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8169d7fe3f0440468b0c9ceda7dfb2de', 'request_id': 'req-67152695-4521-4541-a280-ebe56202ddb4'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:16.275787 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-67152695-4521-4541-a280-ebe56202ddb4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8169d7fe3f0440468b0c9ceda7dfb2de', 'request_id': 'req-67152695-4521-4541-a280-ebe56202ddb4'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:16.288838 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-67152695-4521-4541-a280-ebe56202ddb4 admin admin] The token cache is being invalidate because project 8169d7fe3f0440468b0c9ceda7dfb2de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80154) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:44:16.289130 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-67152695-4521-4541-a280-ebe56202ddb4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:16.289761 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-67152695-4521-4541-a280-ebe56202ddb4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:16.291401 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1454/2908] 10.222.0.64 () {58 vars in 1233 bytes} [Tue Aug 2 10:44:16 2022] DELETE /identity/v3/projects/8169d7fe3f0440468b0c9ceda7dfb2de => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:44:20.524922 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549483e6-b4ed-49c9-b215-e719be86545b service neutron] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:20.526352 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-549483e6-b4ed-49c9-b215-e719be86545b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:44:20.582783 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549483e6-b4ed-49c9-b215-e719be86545b service neutron] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:20.630496 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e97183b-9853-4fbd-b28c-7891dc1faa6f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d7de62b9ee4039bdc79627ac7fb499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:20.631108 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e97183b-9853-4fbd-b28c-7891dc1faa6f service cinder] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:20.631394 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e97183b-9853-4fbd-b28c-7891dc1faa6f service cinder] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:20.631671 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e97183b-9853-4fbd-b28c-7891dc1faa6f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:20.633430 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-7e97183b-9853-4fbd-b28c-7891dc1faa6f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:44:20.639303 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:44:20.698235 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e97183b-9853-4fbd-b28c-7891dc1faa6f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:20.699686 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e97183b-9853-4fbd-b28c-7891dc1faa6f service cinder] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:20.709161 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1455/2909] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:44:20 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:44:26.285586 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=37,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:44:30.705022 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=26 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:44:31.038379 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67152695-4521-4541-a280-ebe56202ddb4 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:31.039820 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-67152695-4521-4541-a280-ebe56202ddb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:44:31.104005 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67152695-4521-4541-a280-ebe56202ddb4 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:31.108563 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10034161-feca-4b84-a6be-9b1c3a89abf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:31.109194 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10034161-feca-4b84-a6be-9b1c3a89abf0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:31.109483 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10034161-feca-4b84-a6be-9b1c3a89abf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:31.109757 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10034161-feca-4b84-a6be-9b1c3a89abf0 admin admin] PATH_INFO: `/v3/users/627617e0c3c6410f88b1bdb55856e7bf` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:31.110905 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10034161-feca-4b84-a6be-9b1c3a89abf0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=627617e0c3c6410f88b1bdb55856e7bf)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:31.112857 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10034161-feca-4b84-a6be-9b1c3a89abf0 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:31.158093 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-10034161-feca-4b84-a6be-9b1c3a89abf0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '627617e0c3c6410f88b1bdb55856e7bf'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:31.164775 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-10034161-feca-4b84-a6be-9b1c3a89abf0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '627617e0c3c6410f88b1bdb55856e7bf'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:31.166077 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-10034161-feca-4b84-a6be-9b1c3a89abf0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '627617e0c3c6410f88b1bdb55856e7bf'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:31.174399 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-10034161-feca-4b84-a6be-9b1c3a89abf0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '627617e0c3c6410f88b1bdb55856e7bf'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:31.186388 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-10034161-feca-4b84-a6be-9b1c3a89abf0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '627617e0c3c6410f88b1bdb55856e7bf'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:31.192222 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1455/2910] 10.222.0.64 () {58 vars in 1224 bytes} [Tue Aug 2 10:44:31 2022] DELETE /identity/v3/users/627617e0c3c6410f88b1bdb55856e7bf => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:44:31.197981 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6653196-ca70-4975-a46c-c9dc184b2035 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:31.198277 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6653196-ca70-4975-a46c-c9dc184b2035 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:31.198558 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6653196-ca70-4975-a46c-c9dc184b2035 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:31.217929 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-b6653196-ca70-4975-a46c-c9dc184b2035 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:44:31.219143 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-b6653196-ca70-4975-a46c-c9dc184b2035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:44:31.280400 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1456/2911] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:44:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:44:31.290254 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10034161-feca-4b84-a6be-9b1c3a89abf0 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:31.291706 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-10034161-feca-4b84-a6be-9b1c3a89abf0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:44:31.354911 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10034161-feca-4b84-a6be-9b1c3a89abf0 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:31.358621 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389c5bb0-2da1-4ca8-be18-32cc9fbf131d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:31.359237 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389c5bb0-2da1-4ca8-be18-32cc9fbf131d service neutron] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:31.359520 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389c5bb0-2da1-4ca8-be18-32cc9fbf131d service neutron] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:31.359795 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389c5bb0-2da1-4ca8-be18-32cc9fbf131d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:31.369276 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389c5bb0-2da1-4ca8-be18-32cc9fbf131d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:31.370707 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389c5bb0-2da1-4ca8-be18-32cc9fbf131d service neutron] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:31.380372 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1456/2912] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:44:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:44:31.660059 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6653196-ca70-4975-a46c-c9dc184b2035 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:31.661634 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-b6653196-ca70-4975-a46c-c9dc184b2035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:44:31.675365 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6653196-ca70-4975-a46c-c9dc184b2035 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:31.680539 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40309c51-8f0d-426c-884d-612a985214ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:31.682528 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40309c51-8f0d-426c-884d-612a985214ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:31.682880 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40309c51-8f0d-426c-884d-612a985214ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:31.683420 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40309c51-8f0d-426c-884d-612a985214ed admin admin] PATH_INFO: `/v3/projects/3efda2dff4784b37b047acc487385e3e` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:31.684711 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40309c51-8f0d-426c-884d-612a985214ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=3efda2dff4784b37b047acc487385e3e)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:31.686596 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40309c51-8f0d-426c-884d-612a985214ed admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:31.735939 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-40309c51-8f0d-426c-884d-612a985214ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3efda2dff4784b37b047acc487385e3e', 'request_id': 'req-40309c51-8f0d-426c-884d-612a985214ed'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:31.741512 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-40309c51-8f0d-426c-884d-612a985214ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3efda2dff4784b37b047acc487385e3e', 'request_id': 'req-40309c51-8f0d-426c-884d-612a985214ed'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:31.748357 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-40309c51-8f0d-426c-884d-612a985214ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3efda2dff4784b37b047acc487385e3e', 'request_id': 'req-40309c51-8f0d-426c-884d-612a985214ed'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:31.761648 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-40309c51-8f0d-426c-884d-612a985214ed admin admin] The token cache is being invalidate because project 3efda2dff4784b37b047acc487385e3e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80153) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:44:31.761952 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-40309c51-8f0d-426c-884d-612a985214ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:31.762567 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-40309c51-8f0d-426c-884d-612a985214ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:31.764116 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1457/2913] 10.222.0.64 () {58 vars in 1233 bytes} [Tue Aug 2 10:44:31 2022] DELETE /identity/v3/projects/3efda2dff4784b37b047acc487385e3e => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:44:31.768289 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389c5bb0-2da1-4ca8-be18-32cc9fbf131d service neutron] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:31.777893 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389c5bb0-2da1-4ca8-be18-32cc9fbf131d service neutron] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:31.796352 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:44:31.838789 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aef20c1-0531-4320-acf0-09d806e77642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:31.839707 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aef20c1-0531-4320-acf0-09d806e77642 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:31.840015 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aef20c1-0531-4320-acf0-09d806e77642 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:31.840320 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aef20c1-0531-4320-acf0-09d806e77642 admin admin] PATH_INFO: `/v3/users/9ac459e8d09b4c1ba9c7b5289feb234d` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:31.841685 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aef20c1-0531-4320-acf0-09d806e77642 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ac459e8d09b4c1ba9c7b5289feb234d)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:31.843613 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aef20c1-0531-4320-acf0-09d806e77642 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:31.890958 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-2aef20c1-0531-4320-acf0-09d806e77642 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ac459e8d09b4c1ba9c7b5289feb234d'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:31.900994 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-2aef20c1-0531-4320-acf0-09d806e77642 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ac459e8d09b4c1ba9c7b5289feb234d'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:31.901645 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-2aef20c1-0531-4320-acf0-09d806e77642 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ac459e8d09b4c1ba9c7b5289feb234d'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:31.910868 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-2aef20c1-0531-4320-acf0-09d806e77642 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ac459e8d09b4c1ba9c7b5289feb234d'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:31.924093 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-2aef20c1-0531-4320-acf0-09d806e77642 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ac459e8d09b4c1ba9c7b5289feb234d'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:31.930915 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1457/2914] 10.222.0.64 () {58 vars in 1224 bytes} [Tue Aug 2 10:44:31 2022] DELETE /identity/v3/users/9ac459e8d09b4c1ba9c7b5289feb234d => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:44:32.203618 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40309c51-8f0d-426c-884d-612a985214ed admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:32.205397 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-40309c51-8f0d-426c-884d-612a985214ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:44:32.224284 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40309c51-8f0d-426c-884d-612a985214ed admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:32.286832 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7bee38-d5c6-4fb2-a8a8-315321de174c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:32.287422 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7bee38-d5c6-4fb2-a8a8-315321de174c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:32.287707 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7bee38-d5c6-4fb2-a8a8-315321de174c admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:32.287986 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7bee38-d5c6-4fb2-a8a8-315321de174c admin admin] PATH_INFO: `/v3/projects/761322b96375471aa8595ee7c876e767` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:32.289175 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7bee38-d5c6-4fb2-a8a8-315321de174c admin admin] RBAC: Authorizing `identity:delete_project(project_id=761322b96375471aa8595ee7c876e767)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:32.291118 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7bee38-d5c6-4fb2-a8a8-315321de174c admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:32.322352 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:44:32.341446 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-3c7bee38-d5c6-4fb2-a8a8-315321de174c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '761322b96375471aa8595ee7c876e767', 'request_id': 'req-3c7bee38-d5c6-4fb2-a8a8-315321de174c'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:32.346823 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-3c7bee38-d5c6-4fb2-a8a8-315321de174c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '761322b96375471aa8595ee7c876e767', 'request_id': 'req-3c7bee38-d5c6-4fb2-a8a8-315321de174c'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:32.353173 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-3c7bee38-d5c6-4fb2-a8a8-315321de174c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '761322b96375471aa8595ee7c876e767', 'request_id': 'req-3c7bee38-d5c6-4fb2-a8a8-315321de174c'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:32.366760 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-3c7bee38-d5c6-4fb2-a8a8-315321de174c admin admin] The token cache is being invalidate because project 761322b96375471aa8595ee7c876e767 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80153) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:44:32.367094 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-3c7bee38-d5c6-4fb2-a8a8-315321de174c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:32.367749 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-3c7bee38-d5c6-4fb2-a8a8-315321de174c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:32.369909 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1458/2915] 10.222.0.64 () {58 vars in 1233 bytes} [Tue Aug 2 10:44:32 2022] DELETE /identity/v3/projects/761322b96375471aa8595ee7c876e767 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:44:32.375853 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a93bf54-5395-4f66-9ee3-6363df70a4bc None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:32.376333 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a93bf54-5395-4f66-9ee3-6363df70a4bc None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:32.376577 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a93bf54-5395-4f66-9ee3-6363df70a4bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:32.398214 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-7a93bf54-5395-4f66-9ee3-6363df70a4bc None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:44:32.399424 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-7a93bf54-5395-4f66-9ee3-6363df70a4bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:44:32.462626 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1458/2916] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:44:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:44:32.472185 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbcab61-5da2-4170-be1a-996afa8dcbb4 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:32.472513 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbcab61-5da2-4170-be1a-996afa8dcbb4 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:32.472798 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbcab61-5da2-4170-be1a-996afa8dcbb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:32.494791 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-ebbcab61-5da2-4170-be1a-996afa8dcbb4 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:44:32.496107 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-ebbcab61-5da2-4170-be1a-996afa8dcbb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:44:32.504531 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1459/2917] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:44:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:44:32.509522 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a93bf54-5395-4f66-9ee3-6363df70a4bc None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:32.519291 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a93bf54-5395-4f66-9ee3-6363df70a4bc None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:32.523929 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc4b94e-c0d5-44bb-a324-6ebfddaf2d7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:32.524576 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc4b94e-c0d5-44bb-a324-6ebfddaf2d7c admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:32.524864 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc4b94e-c0d5-44bb-a324-6ebfddaf2d7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:32.525141 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc4b94e-c0d5-44bb-a324-6ebfddaf2d7c admin admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:32.525852 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc4b94e-c0d5-44bb-a324-6ebfddaf2d7c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:44:32.526162 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc4b94e-c0d5-44bb-a324-6ebfddaf2d7c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:32.527758 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc4b94e-c0d5-44bb-a324-6ebfddaf2d7c admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:32.539539 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1459/2918] 10.222.0.64 () {58 vars in 1153 bytes} [Tue Aug 2 10:44:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:44:32.544478 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbcab61-5da2-4170-be1a-996afa8dcbb4 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:32.553945 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbcab61-5da2-4170-be1a-996afa8dcbb4 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:32.559237 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9d1c3d-6659-4d50-8012-bf974a51e932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:32.559843 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9d1c3d-6659-4d50-8012-bf974a51e932 admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:32.560128 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9d1c3d-6659-4d50-8012-bf974a51e932 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:32.560434 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9d1c3d-6659-4d50-8012-bf974a51e932 admin admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:32.561109 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9d1c3d-6659-4d50-8012-bf974a51e932 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:32.563005 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9d1c3d-6659-4d50-8012-bf974a51e932 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:32.589895 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1460/2919] 10.222.0.64 () {60 vars in 1153 bytes} [Tue Aug 2 10:44:32 2022] POST /identity/v3/projects => generated 371 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:44:32.594029 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc4b94e-c0d5-44bb-a324-6ebfddaf2d7c admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:32.604719 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc4b94e-c0d5-44bb-a324-6ebfddaf2d7c admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:32.630381 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:44:32.643776 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9d1c3d-6659-4d50-8012-bf974a51e932 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:32.645603 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-dd9d1c3d-6659-4d50-8012-bf974a51e932 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:44:32.663679 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdcf582-85b4-4309-8ef4-0840729ba2d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:32.664328 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdcf582-85b4-4309-8ef4-0840729ba2d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:32.664619 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdcf582-85b4-4309-8ef4-0840729ba2d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:32.664893 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdcf582-85b4-4309-8ef4-0840729ba2d7 admin admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:32.666126 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdcf582-85b4-4309-8ef4-0840729ba2d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:32.668038 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdcf582-85b4-4309-8ef4-0840729ba2d7 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:32.691484 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1460/2920] 10.222.0.64 () {60 vars in 1144 bytes} [Tue Aug 2 10:44:32 2022] POST /identity/v3/users => generated 286 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:44:32.695478 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdcf582-85b4-4309-8ef4-0840729ba2d7 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:32.705715 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdcf582-85b4-4309-8ef4-0840729ba2d7 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:32.710522 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0abd2e-904f-42f6-b403-ea2c60222230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:32.711122 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0abd2e-904f-42f6-b403-ea2c60222230 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:32.711411 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0abd2e-904f-42f6-b403-ea2c60222230 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:32.711696 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0abd2e-904f-42f6-b403-ea2c60222230 admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:32.712994 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0abd2e-904f-42f6-b403-ea2c60222230 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:32.713973 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9d1c3d-6659-4d50-8012-bf974a51e932 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:32.714587 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0abd2e-904f-42f6-b403-ea2c60222230 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:32.718686 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381745ce-baa9-4003-a3ad-b10b746092af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:32.719318 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381745ce-baa9-4003-a3ad-b10b746092af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:32.719600 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381745ce-baa9-4003-a3ad-b10b746092af admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:32.719903 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381745ce-baa9-4003-a3ad-b10b746092af admin admin] PATH_INFO: `/v3/users/ae5cca45c480411cb12758b3785cf1af` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:32.721114 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381745ce-baa9-4003-a3ad-b10b746092af admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae5cca45c480411cb12758b3785cf1af)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:32.723012 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381745ce-baa9-4003-a3ad-b10b746092af admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:32.725576 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1461/2921] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:44:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:44:32.729410 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0abd2e-904f-42f6-b403-ea2c60222230 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:32.739592 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0abd2e-904f-42f6-b403-ea2c60222230 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:32.744576 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c7037a-125c-4b8a-af72-1ba20a0d280b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:32.745223 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c7037a-125c-4b8a-af72-1ba20a0d280b admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:32.745506 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c7037a-125c-4b8a-af72-1ba20a0d280b admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:32.745830 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c7037a-125c-4b8a-af72-1ba20a0d280b admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:32.746534 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c7037a-125c-4b8a-af72-1ba20a0d280b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:32.748312 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c7037a-125c-4b8a-af72-1ba20a0d280b admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:32.760030 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1462/2922] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:44:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:44:32.764142 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c7037a-125c-4b8a-af72-1ba20a0d280b admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:32.773559 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-381745ce-baa9-4003-a3ad-b10b746092af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae5cca45c480411cb12758b3785cf1af'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:32.774051 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c7037a-125c-4b8a-af72-1ba20a0d280b admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:32.779251 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51eadc4e-f32d-40c2-bb6f-1231cd2c17dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:32.779917 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eadc4e-f32d-40c2-bb6f-1231cd2c17dd admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:32.780201 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eadc4e-f32d-40c2-bb6f-1231cd2c17dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:32.780514 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eadc4e-f32d-40c2-bb6f-1231cd2c17dd admin admin] PATH_INFO: `/v3/projects/effb67c7e4314761931d9c436dd65f82/users/1ad58a157665404ba6664b819252232e/roles/66c1074780954f8e87ed11a5c4e8c36a` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:32.781078 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-381745ce-baa9-4003-a3ad-b10b746092af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae5cca45c480411cb12758b3785cf1af'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:32.788781 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-381745ce-baa9-4003-a3ad-b10b746092af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae5cca45c480411cb12758b3785cf1af'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:32.789971 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-381745ce-baa9-4003-a3ad-b10b746092af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae5cca45c480411cb12758b3785cf1af'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:32.790402 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-381745ce-baa9-4003-a3ad-b10b746092af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae5cca45c480411cb12758b3785cf1af'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:32.795887 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51eadc4e-f32d-40c2-bb6f-1231cd2c17dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=effb67c7e4314761931d9c436dd65f82, user_id=1ad58a157665404ba6664b819252232e, role_id=66c1074780954f8e87ed11a5c4e8c36a)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:32.798585 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51eadc4e-f32d-40c2-bb6f-1231cd2c17dd admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:32.809474 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1461/2923] 10.222.0.64 () {58 vars in 1224 bytes} [Tue Aug 2 10:44:32 2022] DELETE /identity/v3/users/ae5cca45c480411cb12758b3785cf1af => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:44:32.813912 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927298c2-0ea3-43f1-be41-9d97afe359c2 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:32.824667 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1463/2924] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 10:44:32 2022] PUT /identity/v3/projects/effb67c7e4314761931d9c436dd65f82/users/1ad58a157665404ba6664b819252232e/roles/66c1074780954f8e87ed11a5c4e8c36a => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:44:32.824790 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927298c2-0ea3-43f1-be41-9d97afe359c2 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:32.824790 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927298c2-0ea3-43f1-be41-9d97afe359c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:32.824790 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:DELETE=16,keystone:SELECT=81,keystone:INSERT=3 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:44:32.826591 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0061b6-0493-41fc-9e9a-e68a74f76128 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:32.826895 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0061b6-0493-41fc-9e9a-e68a74f76128 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:32.827160 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0061b6-0493-41fc-9e9a-e68a74f76128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:32.837034 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-927298c2-0ea3-43f1-be41-9d97afe359c2 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:44:32.839379 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-927298c2-0ea3-43f1-be41-9d97afe359c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:44:32.856007 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-4d0061b6-0493-41fc-9e9a-e68a74f76128 None None] MFA Rules not processed for user `1ad58a157665404ba6664b819252232e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:44:32.857451 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-4d0061b6-0493-41fc-9e9a-e68a74f76128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:44:32.909958 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1462/2925] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:44:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:44:32.919550 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927298c2-0ea3-43f1-be41-9d97afe359c2 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:32.920930 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-927298c2-0ea3-43f1-be41-9d97afe359c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:44:32.925706 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1464/2926] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:44:32 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:44:32.937326 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0061b6-0493-41fc-9e9a-e68a74f76128 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:32.938842 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-4d0061b6-0493-41fc-9e9a-e68a74f76128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:44:32.980167 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927298c2-0ea3-43f1-be41-9d97afe359c2 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:32.983995 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a01250-6c3f-4f03-8068-b6269677874f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:32.984658 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a01250-6c3f-4f03-8068-b6269677874f service neutron] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:32.984947 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a01250-6c3f-4f03-8068-b6269677874f service neutron] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:32.985223 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a01250-6c3f-4f03-8068-b6269677874f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:32.994614 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a01250-6c3f-4f03-8068-b6269677874f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:32.996055 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a01250-6c3f-4f03-8068-b6269677874f service neutron] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:32.996722 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0061b6-0493-41fc-9e9a-e68a74f76128 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:33.000376 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea37c290-fed0-4751-8a84-3eefea2ff1cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d7de62b9ee4039bdc79627ac7fb499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:33.000999 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea37c290-fed0-4751-8a84-3eefea2ff1cd service cinder] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:33.001283 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea37c290-fed0-4751-8a84-3eefea2ff1cd service cinder] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:33.001558 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea37c290-fed0-4751-8a84-3eefea2ff1cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:33.006011 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1463/2927] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:44:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:44:33.010956 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea37c290-fed0-4751-8a84-3eefea2ff1cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:33.012416 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea37c290-fed0-4751-8a84-3eefea2ff1cd service cinder] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:33.021620 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1465/2928] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:44:32 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:44:33.044632 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a01250-6c3f-4f03-8068-b6269677874f service neutron] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:33.046214 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-70a01250-6c3f-4f03-8068-b6269677874f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:44:33.060612 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a01250-6c3f-4f03-8068-b6269677874f service neutron] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:33.065690 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169a2ac3-0914-4d97-b007-59a4b18b06e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:33.066334 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169a2ac3-0914-4d97-b007-59a4b18b06e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:33.066616 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169a2ac3-0914-4d97-b007-59a4b18b06e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:33.066888 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169a2ac3-0914-4d97-b007-59a4b18b06e6 admin admin] PATH_INFO: `/v3/users/1ad58a157665404ba6664b819252232e` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:33.068325 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169a2ac3-0914-4d97-b007-59a4b18b06e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ad58a157665404ba6664b819252232e)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:33.070250 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169a2ac3-0914-4d97-b007-59a4b18b06e6 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:33.119982 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-169a2ac3-0914-4d97-b007-59a4b18b06e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ad58a157665404ba6664b819252232e'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:33.126944 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-169a2ac3-0914-4d97-b007-59a4b18b06e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ad58a157665404ba6664b819252232e'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:33.134487 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-169a2ac3-0914-4d97-b007-59a4b18b06e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ad58a157665404ba6664b819252232e'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:33.135123 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-169a2ac3-0914-4d97-b007-59a4b18b06e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ad58a157665404ba6664b819252232e'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:33.135773 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-169a2ac3-0914-4d97-b007-59a4b18b06e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ad58a157665404ba6664b819252232e'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:33.157683 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1464/2929] 10.222.0.64 () {58 vars in 1224 bytes} [Tue Aug 2 10:44:33 2022] DELETE /identity/v3/users/1ad58a157665404ba6664b819252232e => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:44:33.162194 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1564a3-3525-42a7-bb8f-c7b37a83ccd7 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:33.162489 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1564a3-3525-42a7-bb8f-c7b37a83ccd7 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:33.162782 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1564a3-3525-42a7-bb8f-c7b37a83ccd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:33.171283 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:44:33.184700 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-7a1564a3-3525-42a7-bb8f-c7b37a83ccd7 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:44:33.185929 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-7a1564a3-3525-42a7-bb8f-c7b37a83ccd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:44:33.248214 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1466/2930] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:44:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:44:33.258252 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169a2ac3-0914-4d97-b007-59a4b18b06e6 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:33.259706 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-169a2ac3-0914-4d97-b007-59a4b18b06e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:44:33.287836 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:44:33.297358 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1564a3-3525-42a7-bb8f-c7b37a83ccd7 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:33.298996 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-7a1564a3-3525-42a7-bb8f-c7b37a83ccd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:44:33.312404 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1564a3-3525-42a7-bb8f-c7b37a83ccd7 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:33.317168 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5e0fba-e7cd-4d10-80d1-26a95e869abf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:33.317825 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5e0fba-e7cd-4d10-80d1-26a95e869abf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:33.318106 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5e0fba-e7cd-4d10-80d1-26a95e869abf admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:33.318385 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5e0fba-e7cd-4d10-80d1-26a95e869abf admin admin] PATH_INFO: `/v3/projects/5175365a2db9489986f5a84961a5d0fb` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:33.319806 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5e0fba-e7cd-4d10-80d1-26a95e869abf admin admin] RBAC: Authorizing `identity:delete_project(project_id=5175365a2db9489986f5a84961a5d0fb)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:33.321572 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5e0fba-e7cd-4d10-80d1-26a95e869abf admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:33.337819 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169a2ac3-0914-4d97-b007-59a4b18b06e6 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:33.341758 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a413cc-3104-46be-a8d4-4968fa57f327 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:33.342392 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a413cc-3104-46be-a8d4-4968fa57f327 service neutron] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:33.342681 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a413cc-3104-46be-a8d4-4968fa57f327 service neutron] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:33.342959 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a413cc-3104-46be-a8d4-4968fa57f327 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:33.354257 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a413cc-3104-46be-a8d4-4968fa57f327 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:33.355742 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a413cc-3104-46be-a8d4-4968fa57f327 service neutron] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:33.367554 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1465/2931] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:44:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:44:33.383143 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-2d5e0fba-e7cd-4d10-80d1-26a95e869abf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5175365a2db9489986f5a84961a5d0fb', 'request_id': 'req-2d5e0fba-e7cd-4d10-80d1-26a95e869abf'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:33.390474 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-2d5e0fba-e7cd-4d10-80d1-26a95e869abf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5175365a2db9489986f5a84961a5d0fb', 'request_id': 'req-2d5e0fba-e7cd-4d10-80d1-26a95e869abf'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:33.397212 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-2d5e0fba-e7cd-4d10-80d1-26a95e869abf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5175365a2db9489986f5a84961a5d0fb', 'request_id': 'req-2d5e0fba-e7cd-4d10-80d1-26a95e869abf'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:33.413247 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-2d5e0fba-e7cd-4d10-80d1-26a95e869abf admin admin] The token cache is being invalidate because project 5175365a2db9489986f5a84961a5d0fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80154) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:44:33.413538 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-2d5e0fba-e7cd-4d10-80d1-26a95e869abf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:33.414197 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-2d5e0fba-e7cd-4d10-80d1-26a95e869abf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:33.428353 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1467/2932] 10.222.0.64 () {58 vars in 1233 bytes} [Tue Aug 2 10:44:33 2022] DELETE /identity/v3/projects/5175365a2db9489986f5a84961a5d0fb => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:44:33.679634 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a413cc-3104-46be-a8d4-4968fa57f327 service neutron] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:33.682441 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-81a413cc-3104-46be-a8d4-4968fa57f327 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:44:33.700140 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a413cc-3104-46be-a8d4-4968fa57f327 service neutron] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:33.761594 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff95cfa-52df-4148-9d49-5ead585dbb7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:33.762303 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff95cfa-52df-4148-9d49-5ead585dbb7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:33.762608 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff95cfa-52df-4148-9d49-5ead585dbb7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:33.762890 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff95cfa-52df-4148-9d49-5ead585dbb7b admin admin] PATH_INFO: `/v3/projects/effb67c7e4314761931d9c436dd65f82` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:33.764156 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff95cfa-52df-4148-9d49-5ead585dbb7b admin admin] RBAC: Authorizing `identity:delete_project(project_id=effb67c7e4314761931d9c436dd65f82)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:33.766394 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff95cfa-52df-4148-9d49-5ead585dbb7b admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:33.814002 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-1ff95cfa-52df-4148-9d49-5ead585dbb7b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'effb67c7e4314761931d9c436dd65f82', 'request_id': 'req-1ff95cfa-52df-4148-9d49-5ead585dbb7b'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:33.819358 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-1ff95cfa-52df-4148-9d49-5ead585dbb7b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'effb67c7e4314761931d9c436dd65f82', 'request_id': 'req-1ff95cfa-52df-4148-9d49-5ead585dbb7b'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:33.826052 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-1ff95cfa-52df-4148-9d49-5ead585dbb7b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'effb67c7e4314761931d9c436dd65f82', 'request_id': 'req-1ff95cfa-52df-4148-9d49-5ead585dbb7b'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:33.838186 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-1ff95cfa-52df-4148-9d49-5ead585dbb7b admin admin] The token cache is being invalidate because project effb67c7e4314761931d9c436dd65f82 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80153) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:44:33.838472 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-1ff95cfa-52df-4148-9d49-5ead585dbb7b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:33.839089 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-1ff95cfa-52df-4148-9d49-5ead585dbb7b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:33.840937 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1466/2933] 10.222.0.64 () {58 vars in 1233 bytes} [Tue Aug 2 10:44:33 2022] DELETE /identity/v3/projects/effb67c7e4314761931d9c436dd65f82 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:44:33.846447 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbf21e6-1185-4a56-9149-136a5189f42b None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:33.846812 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbf21e6-1185-4a56-9149-136a5189f42b None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:33.847089 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbf21e6-1185-4a56-9149-136a5189f42b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:33.866614 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-ecbf21e6-1185-4a56-9149-136a5189f42b None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:44:33.867831 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-ecbf21e6-1185-4a56-9149-136a5189f42b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:44:33.931333 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1468/2934] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:44:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:44:33.938925 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36b85ea-c132-43b8-82c5-512f15e98f71 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:33.939217 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36b85ea-c132-43b8-82c5-512f15e98f71 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:33.939496 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36b85ea-c132-43b8-82c5-512f15e98f71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:33.959007 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-d36b85ea-c132-43b8-82c5-512f15e98f71 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:44:33.960258 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-d36b85ea-c132-43b8-82c5-512f15e98f71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:44:33.968712 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1467/2935] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:44:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:44:33.973344 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbf21e6-1185-4a56-9149-136a5189f42b None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:33.983471 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbf21e6-1185-4a56-9149-136a5189f42b None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:33.988035 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e33b5b4-9bc3-4e28-841c-3be6a942ad74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:33.988668 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e33b5b4-9bc3-4e28-841c-3be6a942ad74 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:33.988949 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e33b5b4-9bc3-4e28-841c-3be6a942ad74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:33.989227 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e33b5b4-9bc3-4e28-841c-3be6a942ad74 admin admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:33.989918 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e33b5b4-9bc3-4e28-841c-3be6a942ad74 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:44:33.990223 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e33b5b4-9bc3-4e28-841c-3be6a942ad74 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:33.992346 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e33b5b4-9bc3-4e28-841c-3be6a942ad74 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:34.003754 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1469/2936] 10.222.0.64 () {58 vars in 1153 bytes} [Tue Aug 2 10:44:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:44:34.008293 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36b85ea-c132-43b8-82c5-512f15e98f71 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:34.018105 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36b85ea-c132-43b8-82c5-512f15e98f71 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:34.023004 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9409b0c7-0471-4267-bb25-1cac8e933f33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:34.023609 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9409b0c7-0471-4267-bb25-1cac8e933f33 admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:34.023894 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9409b0c7-0471-4267-bb25-1cac8e933f33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:34.024175 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9409b0c7-0471-4267-bb25-1cac8e933f33 admin admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:34.024900 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9409b0c7-0471-4267-bb25-1cac8e933f33 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:34.026789 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9409b0c7-0471-4267-bb25-1cac8e933f33 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:34.053692 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1468/2937] 10.222.0.64 () {60 vars in 1153 bytes} [Tue Aug 2 10:44:34 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:44:34.058636 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e33b5b4-9bc3-4e28-841c-3be6a942ad74 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:34.068561 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e33b5b4-9bc3-4e28-841c-3be6a942ad74 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:34.083399 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:44:34.130449 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b34120-f017-480d-be5f-874d94e7a899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:34.130989 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b34120-f017-480d-be5f-874d94e7a899 admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:34.131272 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b34120-f017-480d-be5f-874d94e7a899 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:34.131548 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b34120-f017-480d-be5f-874d94e7a899 admin admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:34.132209 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b34120-f017-480d-be5f-874d94e7a899 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:34.134140 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b34120-f017-480d-be5f-874d94e7a899 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:34.156680 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1470/2938] 10.222.0.64 () {60 vars in 1144 bytes} [Tue Aug 2 10:44:34 2022] POST /identity/v3/users => generated 292 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:44:34.160731 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9409b0c7-0471-4267-bb25-1cac8e933f33 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:34.171504 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9409b0c7-0471-4267-bb25-1cac8e933f33 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:34.176196 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8b111c-062b-4559-85c6-629901029d72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:34.176828 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8b111c-062b-4559-85c6-629901029d72 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:34.177110 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8b111c-062b-4559-85c6-629901029d72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:34.177386 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8b111c-062b-4559-85c6-629901029d72 admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:34.178073 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8b111c-062b-4559-85c6-629901029d72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:34.179921 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8b111c-062b-4559-85c6-629901029d72 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:34.189827 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1469/2939] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:44:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:44:34.193650 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b34120-f017-480d-be5f-874d94e7a899 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:34.204212 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b34120-f017-480d-be5f-874d94e7a899 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:34.208811 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744cc5de-018f-4017-823f-e3a597f00280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:34.209419 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744cc5de-018f-4017-823f-e3a597f00280 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:34.209704 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744cc5de-018f-4017-823f-e3a597f00280 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:34.209982 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744cc5de-018f-4017-823f-e3a597f00280 admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:34.210678 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744cc5de-018f-4017-823f-e3a597f00280 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:34.212416 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744cc5de-018f-4017-823f-e3a597f00280 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:34.222972 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1471/2940] 10.222.0.64 () {58 vars in 1122 bytes} [Tue Aug 2 10:44:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:44:34.227335 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8b111c-062b-4559-85c6-629901029d72 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:34.237624 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8b111c-062b-4559-85c6-629901029d72 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:34.242528 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bb7960-ddd6-460c-af28-6ae9d6ba8f12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:34.243193 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bb7960-ddd6-460c-af28-6ae9d6ba8f12 admin admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:34.243558 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bb7960-ddd6-460c-af28-6ae9d6ba8f12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:34.243836 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bb7960-ddd6-460c-af28-6ae9d6ba8f12 admin admin] PATH_INFO: `/v3/projects/eb5cf7080afb4185986a40930a4d0c18/users/e6e812439daa46d0adfe7b546ab6d567/roles/66c1074780954f8e87ed11a5c4e8c36a` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:34.256055 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:44:34.261468 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bb7960-ddd6-460c-af28-6ae9d6ba8f12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb5cf7080afb4185986a40930a4d0c18, user_id=e6e812439daa46d0adfe7b546ab6d567, role_id=66c1074780954f8e87ed11a5c4e8c36a)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:34.264176 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bb7960-ddd6-460c-af28-6ae9d6ba8f12 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:34.278932 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1470/2941] 10.222.0.64 () {60 vars in 1483 bytes} [Tue Aug 2 10:44:34 2022] PUT /identity/v3/projects/eb5cf7080afb4185986a40930a4d0c18/users/e6e812439daa46d0adfe7b546ab6d567/roles/66c1074780954f8e87ed11a5c4e8c36a => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:44:34.289774 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a84278f-1700-4715-a92e-3e7fb6e89b96 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:34.290113 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a84278f-1700-4715-a92e-3e7fb6e89b96 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:34.290393 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a84278f-1700-4715-a92e-3e7fb6e89b96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:34.317511 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-3a84278f-1700-4715-a92e-3e7fb6e89b96 None None] MFA Rules not processed for user `e6e812439daa46d0adfe7b546ab6d567`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:44:34.318732 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-3a84278f-1700-4715-a92e-3e7fb6e89b96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:44:34.382042 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1472/2942] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:44:34 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:44:34.393880 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bb7960-ddd6-460c-af28-6ae9d6ba8f12 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:34.395585 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-03bb7960-ddd6-460c-af28-6ae9d6ba8f12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:44:34.451803 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bb7960-ddd6-460c-af28-6ae9d6ba8f12 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:34.455467 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88075fd-c4f2-4272-af2e-50fb780ac726 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d7de62b9ee4039bdc79627ac7fb499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:34.456103 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88075fd-c4f2-4272-af2e-50fb780ac726 service cinder] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:34.456418 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88075fd-c4f2-4272-af2e-50fb780ac726 service cinder] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:34.456729 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88075fd-c4f2-4272-af2e-50fb780ac726 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:34.465658 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88075fd-c4f2-4272-af2e-50fb780ac726 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:34.467094 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88075fd-c4f2-4272-af2e-50fb780ac726 service cinder] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:34.476107 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1471/2943] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:44:34 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:44:38.076685 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a84278f-1700-4715-a92e-3e7fb6e89b96 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:38.139188 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a84278f-1700-4715-a92e-3e7fb6e89b96 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:38.143768 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa39766d-2299-4204-a30b-484393a99a4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:38.144390 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa39766d-2299-4204-a30b-484393a99a4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:38.144668 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa39766d-2299-4204-a30b-484393a99a4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:38.144943 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa39766d-2299-4204-a30b-484393a99a4c admin admin] PATH_INFO: `/v3/users/e6e812439daa46d0adfe7b546ab6d567` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:38.146103 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa39766d-2299-4204-a30b-484393a99a4c admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6e812439daa46d0adfe7b546ab6d567)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:38.148021 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa39766d-2299-4204-a30b-484393a99a4c admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:38.176053 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:44:38.195813 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-fa39766d-2299-4204-a30b-484393a99a4c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6e812439daa46d0adfe7b546ab6d567'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:38.202600 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-fa39766d-2299-4204-a30b-484393a99a4c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6e812439daa46d0adfe7b546ab6d567'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:38.203257 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-fa39766d-2299-4204-a30b-484393a99a4c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6e812439daa46d0adfe7b546ab6d567'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:38.210565 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-fa39766d-2299-4204-a30b-484393a99a4c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6e812439daa46d0adfe7b546ab6d567'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:38.222829 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-fa39766d-2299-4204-a30b-484393a99a4c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6e812439daa46d0adfe7b546ab6d567'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:38.229375 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1473/2944] 10.222.0.64 () {58 vars in 1224 bytes} [Tue Aug 2 10:44:38 2022] DELETE /identity/v3/users/e6e812439daa46d0adfe7b546ab6d567 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:44:38.233991 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279dfc5e-50fc-4736-a082-ab0ad7c31bb0 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:38.234474 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279dfc5e-50fc-4736-a082-ab0ad7c31bb0 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:38.234560 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279dfc5e-50fc-4736-a082-ab0ad7c31bb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:38.253665 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-279dfc5e-50fc-4736-a082-ab0ad7c31bb0 None None] MFA Rules not processed for user `5c5f9105c00b4d00a6249c76315d5e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:44:38.254926 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-279dfc5e-50fc-4736-a082-ab0ad7c31bb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:44:38.313643 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1472/2945] 10.222.0.64 () {58 vars in 958 bytes} [Tue Aug 2 10:44:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:44:38.323159 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa39766d-2299-4204-a30b-484393a99a4c admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:38.324588 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-fa39766d-2299-4204-a30b-484393a99a4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:44:38.381911 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa39766d-2299-4204-a30b-484393a99a4c admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:38.386159 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4866ecd-916e-4b33-8f64-92c5712b6ce6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:38.386783 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4866ecd-916e-4b33-8f64-92c5712b6ce6 service neutron] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:38.387071 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4866ecd-916e-4b33-8f64-92c5712b6ce6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:38.387352 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4866ecd-916e-4b33-8f64-92c5712b6ce6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:38.396889 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4866ecd-916e-4b33-8f64-92c5712b6ce6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:38.398248 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4866ecd-916e-4b33-8f64-92c5712b6ce6 service neutron] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:38.408318 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1474/2946] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:44:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:44:38.681816 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279dfc5e-50fc-4736-a082-ab0ad7c31bb0 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:38.683504 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-279dfc5e-50fc-4736-a082-ab0ad7c31bb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:44:38.702629 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279dfc5e-50fc-4736-a082-ab0ad7c31bb0 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:38.710669 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c4a597-d213-4838-b2dc-f37e67410d5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:38.711636 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c4a597-d213-4838-b2dc-f37e67410d5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:38.711846 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c4a597-d213-4838-b2dc-f37e67410d5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:38.712744 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c4a597-d213-4838-b2dc-f37e67410d5b admin admin] PATH_INFO: `/v3/projects/eb5cf7080afb4185986a40930a4d0c18` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:38.713930 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c4a597-d213-4838-b2dc-f37e67410d5b admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb5cf7080afb4185986a40930a4d0c18)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:38.716044 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c4a597-d213-4838-b2dc-f37e67410d5b admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:38.769377 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-86c4a597-d213-4838-b2dc-f37e67410d5b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb5cf7080afb4185986a40930a4d0c18', 'request_id': 'req-86c4a597-d213-4838-b2dc-f37e67410d5b'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:38.775028 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-86c4a597-d213-4838-b2dc-f37e67410d5b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb5cf7080afb4185986a40930a4d0c18', 'request_id': 'req-86c4a597-d213-4838-b2dc-f37e67410d5b'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:38.786184 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-86c4a597-d213-4838-b2dc-f37e67410d5b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb5cf7080afb4185986a40930a4d0c18', 'request_id': 'req-86c4a597-d213-4838-b2dc-f37e67410d5b'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:38.800757 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-86c4a597-d213-4838-b2dc-f37e67410d5b admin admin] The token cache is being invalidate because project eb5cf7080afb4185986a40930a4d0c18 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80153) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:44:38.801067 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-86c4a597-d213-4838-b2dc-f37e67410d5b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:38.801725 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-86c4a597-d213-4838-b2dc-f37e67410d5b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:44:38.803649 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1473/2947] 10.222.0.64 () {58 vars in 1233 bytes} [Tue Aug 2 10:44:38 2022] DELETE /identity/v3/projects/eb5cf7080afb4185986a40930a4d0c18 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:44:39.127546 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4866ecd-916e-4b33-8f64-92c5712b6ce6 service neutron] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:39.136444 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4866ecd-916e-4b33-8f64-92c5712b6ce6 service neutron] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:39.187145 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62130e5-09e6-47db-bed3-fdd57b20c391 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:39.187678 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62130e5-09e6-47db-bed3-fdd57b20c391 service neutron] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:39.187961 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62130e5-09e6-47db-bed3-fdd57b20c391 service neutron] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:39.188235 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62130e5-09e6-47db-bed3-fdd57b20c391 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:39.190046 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-b62130e5-09e6-47db-bed3-fdd57b20c391 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:44:39.199484 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62130e5-09e6-47db-bed3-fdd57b20c391 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:39.200945 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62130e5-09e6-47db-bed3-fdd57b20c391 service neutron] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:39.208867 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1475/2948] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:44:39 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:44:39.659953 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c4a597-d213-4838-b2dc-f37e67410d5b admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:39.661501 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-86c4a597-d213-4838-b2dc-f37e67410d5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:44:39.693793 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:44:39.750534 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c4a597-d213-4838-b2dc-f37e67410d5b admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:39.755702 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbcbff5-2a4a-4b83-876c-7da05e735d3d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b770f073c86b4d88837a8b1f796a8b1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:39.756357 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbcbff5-2a4a-4b83-876c-7da05e735d3d service placement] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:39.756645 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbcbff5-2a4a-4b83-876c-7da05e735d3d service placement] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:39.756919 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbcbff5-2a4a-4b83-876c-7da05e735d3d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:39.758650 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-cfbcbff5-2a4a-4b83-876c-7da05e735d3d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:44:39.771010 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbcbff5-2a4a-4b83-876c-7da05e735d3d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:39.772613 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbcbff5-2a4a-4b83-876c-7da05e735d3d service placement] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:39.783448 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1474/2949] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:44:39 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:44:41.529779 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62130e5-09e6-47db-bed3-fdd57b20c391 service neutron] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:41.531911 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-b62130e5-09e6-47db-bed3-fdd57b20c391 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:44:41.543488 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62130e5-09e6-47db-bed3-fdd57b20c391 service neutron] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:41.547968 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f62e2d-0989-4cbb-9633-b0ec592ae87e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:41.548645 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f62e2d-0989-4cbb-9633-b0ec592ae87e service neutron] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:41.548932 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f62e2d-0989-4cbb-9633-b0ec592ae87e service neutron] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:41.549210 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f62e2d-0989-4cbb-9633-b0ec592ae87e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:41.551105 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-d2f62e2d-0989-4cbb-9633-b0ec592ae87e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:44:41.564183 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f62e2d-0989-4cbb-9633-b0ec592ae87e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:41.565891 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f62e2d-0989-4cbb-9633-b0ec592ae87e service neutron] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:41.575998 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1476/2950] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:44:41 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:44:42.200346 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbcbff5-2a4a-4b83-876c-7da05e735d3d service placement] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:42.202011 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-cfbcbff5-2a4a-4b83-876c-7da05e735d3d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:44:42.297000 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbcbff5-2a4a-4b83-876c-7da05e735d3d service placement] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:42.302720 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df403994-ba72-42dc-ad3e-ea48363445c2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99655e1f9ca24472a86cd26b0dbd1fc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:42.303607 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df403994-ba72-42dc-ad3e-ea48363445c2 service nova] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:42.303908 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df403994-ba72-42dc-ad3e-ea48363445c2 service nova] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:42.304187 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df403994-ba72-42dc-ad3e-ea48363445c2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:42.305995 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-df403994-ba72-42dc-ad3e-ea48363445c2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:44:42.318929 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df403994-ba72-42dc-ad3e-ea48363445c2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:42.320580 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df403994-ba72-42dc-ad3e-ea48363445c2 service nova] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:42.333269 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1475/2951] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:44:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:44:51.353154 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f62e2d-0989-4cbb-9633-b0ec592ae87e service neutron] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:51.354777 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-d2f62e2d-0989-4cbb-9633-b0ec592ae87e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:44:51.399724 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:44:51.420651 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f62e2d-0989-4cbb-9633-b0ec592ae87e service neutron] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:51.424221 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb465e1-fb57-4ad4-9b32-ee39a912b794 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d7de62b9ee4039bdc79627ac7fb499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:51.424869 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb465e1-fb57-4ad4-9b32-ee39a912b794 service cinder] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:51.425146 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb465e1-fb57-4ad4-9b32-ee39a912b794 service cinder] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:51.425416 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb465e1-fb57-4ad4-9b32-ee39a912b794 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:51.427352 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-aeb465e1-fb57-4ad4-9b32-ee39a912b794 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:44:51.507602 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb465e1-fb57-4ad4-9b32-ee39a912b794 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:51.508983 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb465e1-fb57-4ad4-9b32-ee39a912b794 service cinder] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:51.527202 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1477/2952] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:44:51 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:44:52.326909 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=59 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:44:54.746550 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df403994-ba72-42dc-ad3e-ea48363445c2 service nova] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:54.748083 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-df403994-ba72-42dc-ad3e-ea48363445c2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:44:54.763130 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df403994-ba72-42dc-ad3e-ea48363445c2 service nova] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:54.768996 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702ae8f0-2202-4597-80c1-ee21f2a7b4a3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99655e1f9ca24472a86cd26b0dbd1fc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:54.769796 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702ae8f0-2202-4597-80c1-ee21f2a7b4a3 service nova] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:54.769892 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702ae8f0-2202-4597-80c1-ee21f2a7b4a3 service nova] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:54.770160 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702ae8f0-2202-4597-80c1-ee21f2a7b4a3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:54.772043 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-702ae8f0-2202-4597-80c1-ee21f2a7b4a3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:44:54.788292 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702ae8f0-2202-4597-80c1-ee21f2a7b4a3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:54.789909 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702ae8f0-2202-4597-80c1-ee21f2a7b4a3 service nova] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:54.802544 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1476/2953] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:44:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:44:58.631141 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb465e1-fb57-4ad4-9b32-ee39a912b794 service cinder] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:44:58.632740 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-aeb465e1-fb57-4ad4-9b32-ee39a912b794 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:44:58.644844 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb465e1-fb57-4ad4-9b32-ee39a912b794 service cinder] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:44:58.648519 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba05493-b12a-4b3d-a781-be42aba5f305 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:44:58.649121 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba05493-b12a-4b3d-a781-be42aba5f305 service neutron] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:44:58.649403 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba05493-b12a-4b3d-a781-be42aba5f305 service neutron] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:44:58.649882 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba05493-b12a-4b3d-a781-be42aba5f305 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:44:58.651556 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-eba05493-b12a-4b3d-a781-be42aba5f305 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:44:58.715252 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba05493-b12a-4b3d-a781-be42aba5f305 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:44:58.716722 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba05493-b12a-4b3d-a781-be42aba5f305 service neutron] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:44:58.726065 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1478/2954] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:44:58 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:45:00.551332 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702ae8f0-2202-4597-80c1-ee21f2a7b4a3 service nova] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:45:00.564028 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702ae8f0-2202-4597-80c1-ee21f2a7b4a3 service nova] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:45:00.570237 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb411f2-cd66-43f7-8979-6514107c113d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99655e1f9ca24472a86cd26b0dbd1fc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:45:00.570954 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb411f2-cd66-43f7-8979-6514107c113d service nova] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:45:00.571352 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb411f2-cd66-43f7-8979-6514107c113d service nova] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:45:00.571730 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb411f2-cd66-43f7-8979-6514107c113d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:45:00.573639 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-cdb411f2-cd66-43f7-8979-6514107c113d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:45:00.586830 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb411f2-cd66-43f7-8979-6514107c113d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:45:00.588539 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb411f2-cd66-43f7-8979-6514107c113d service nova] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:45:00.599474 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1477/2955] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:45:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:45:08.721816 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=75 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:45:10.594795 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=12 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:45:52.031791 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba05493-b12a-4b3d-a781-be42aba5f305 service neutron] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:45:52.043236 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba05493-b12a-4b3d-a781-be42aba5f305 service neutron] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:45:52.047028 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c24fc67-b6db-4e1f-8660-0b21165f0fc1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:45:52.047659 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c24fc67-b6db-4e1f-8660-0b21165f0fc1 service neutron] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:45:52.047942 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c24fc67-b6db-4e1f-8660-0b21165f0fc1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:45:52.048222 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c24fc67-b6db-4e1f-8660-0b21165f0fc1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:45:52.050100 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-0c24fc67-b6db-4e1f-8660-0b21165f0fc1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:45:52.116164 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c24fc67-b6db-4e1f-8660-0b21165f0fc1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:45:52.117658 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c24fc67-b6db-4e1f-8660-0b21165f0fc1 service neutron] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:45:52.128684 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1479/2956] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:45:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:45:58.408468 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb411f2-cd66-43f7-8979-6514107c113d service nova] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:45:58.410178 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-cdb411f2-cd66-43f7-8979-6514107c113d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:45:58.436410 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb411f2-cd66-43f7-8979-6514107c113d service nova] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:45:58.441349 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936e95f9-039a-48f5-ac19-b7febf5fa07d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:45:58.442182 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936e95f9-039a-48f5-ac19-b7febf5fa07d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:45:58.442465 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936e95f9-039a-48f5-ac19-b7febf5fa07d admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:45:58.442741 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936e95f9-039a-48f5-ac19-b7febf5fa07d admin admin] PATH_INFO: `/v3/users/7739a00c6524467d8fd139183c6667cf` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:45:58.443918 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936e95f9-039a-48f5-ac19-b7febf5fa07d admin admin] RBAC: Authorizing `identity:delete_user(user_id=7739a00c6524467d8fd139183c6667cf)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:45:58.445994 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936e95f9-039a-48f5-ac19-b7febf5fa07d admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:45:58.504824 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-936e95f9-039a-48f5-ac19-b7febf5fa07d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7739a00c6524467d8fd139183c6667cf'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:45:58.513367 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-936e95f9-039a-48f5-ac19-b7febf5fa07d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7739a00c6524467d8fd139183c6667cf'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:45:58.521525 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-936e95f9-039a-48f5-ac19-b7febf5fa07d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7739a00c6524467d8fd139183c6667cf'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:45:58.522187 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-936e95f9-039a-48f5-ac19-b7febf5fa07d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7739a00c6524467d8fd139183c6667cf'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:45:58.522847 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-936e95f9-039a-48f5-ac19-b7febf5fa07d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7739a00c6524467d8fd139183c6667cf'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:45:58.543422 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1478/2957] 10.222.0.64 () {58 vars in 1224 bytes} [Tue Aug 2 10:45:58 2022] DELETE /identity/v3/users/7739a00c6524467d8fd139183c6667cf => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:45:58.725410 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c24fc67-b6db-4e1f-8660-0b21165f0fc1 service neutron] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:45:58.736581 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c24fc67-b6db-4e1f-8660-0b21165f0fc1 service neutron] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:45:58.797893 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daf9859-a2a1-4c9c-89a0-08cb596e6067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:45:58.798530 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daf9859-a2a1-4c9c-89a0-08cb596e6067 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:45:58.798815 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daf9859-a2a1-4c9c-89a0-08cb596e6067 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:45:58.799090 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daf9859-a2a1-4c9c-89a0-08cb596e6067 admin admin] PATH_INFO: `/v3/projects/0ef2c95a509c446e870129685685e1ac` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:45:58.800425 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5daf9859-a2a1-4c9c-89a0-08cb596e6067 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ef2c95a509c446e870129685685e1ac)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:45:58.802387 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5daf9859-a2a1-4c9c-89a0-08cb596e6067 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:45:58.848867 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-5daf9859-a2a1-4c9c-89a0-08cb596e6067 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ef2c95a509c446e870129685685e1ac', 'request_id': 'req-5daf9859-a2a1-4c9c-89a0-08cb596e6067'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:45:58.854966 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-5daf9859-a2a1-4c9c-89a0-08cb596e6067 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ef2c95a509c446e870129685685e1ac', 'request_id': 'req-5daf9859-a2a1-4c9c-89a0-08cb596e6067'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:45:58.861128 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-5daf9859-a2a1-4c9c-89a0-08cb596e6067 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ef2c95a509c446e870129685685e1ac', 'request_id': 'req-5daf9859-a2a1-4c9c-89a0-08cb596e6067'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:45:58.873707 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-5daf9859-a2a1-4c9c-89a0-08cb596e6067 admin admin] The token cache is being invalidate because project 0ef2c95a509c446e870129685685e1ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80154) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:45:58.874080 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-5daf9859-a2a1-4c9c-89a0-08cb596e6067 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:45:58.874634 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-5daf9859-a2a1-4c9c-89a0-08cb596e6067 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:45:58.876157 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1480/2958] 10.222.0.64 () {58 vars in 1233 bytes} [Tue Aug 2 10:45:58 2022] DELETE /identity/v3/projects/0ef2c95a509c446e870129685685e1ac => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:45:58.880276 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936e95f9-039a-48f5-ac19-b7febf5fa07d admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:45:58.881684 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-936e95f9-039a-48f5-ac19-b7febf5fa07d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:45:58.894944 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936e95f9-039a-48f5-ac19-b7febf5fa07d admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:45:58.952721 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f39f762-b8a2-47d5-99b9-41a555e8eb55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:45:58.953278 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f39f762-b8a2-47d5-99b9-41a555e8eb55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:45:58.953561 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f39f762-b8a2-47d5-99b9-41a555e8eb55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:45:58.953835 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f39f762-b8a2-47d5-99b9-41a555e8eb55 admin admin] PATH_INFO: `/v3/users/dc2ef0f65b6643eca00c2106f8f8c5d5` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:45:58.967882 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f39f762-b8a2-47d5-99b9-41a555e8eb55 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc2ef0f65b6643eca00c2106f8f8c5d5)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:45:58.970038 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f39f762-b8a2-47d5-99b9-41a555e8eb55 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:45:59.028121 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-7f39f762-b8a2-47d5-99b9-41a555e8eb55 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc2ef0f65b6643eca00c2106f8f8c5d5'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:45:59.034867 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-7f39f762-b8a2-47d5-99b9-41a555e8eb55 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc2ef0f65b6643eca00c2106f8f8c5d5'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:45:59.042371 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-7f39f762-b8a2-47d5-99b9-41a555e8eb55 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc2ef0f65b6643eca00c2106f8f8c5d5'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:45:59.043079 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-7f39f762-b8a2-47d5-99b9-41a555e8eb55 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc2ef0f65b6643eca00c2106f8f8c5d5'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:45:59.043728 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-7f39f762-b8a2-47d5-99b9-41a555e8eb55 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc2ef0f65b6643eca00c2106f8f8c5d5'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:45:59.044969 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:45:59.064063 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1479/2959] 10.222.0.64 () {58 vars in 1224 bytes} [Tue Aug 2 10:45:58 2022] DELETE /identity/v3/users/dc2ef0f65b6643eca00c2106f8f8c5d5 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:45:59.241188 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daf9859-a2a1-4c9c-89a0-08cb596e6067 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:45:59.242651 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-5daf9859-a2a1-4c9c-89a0-08cb596e6067 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:45:59.256956 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daf9859-a2a1-4c9c-89a0-08cb596e6067 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:45:59.286280 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:45:59.325957 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594382a0-6646-46b4-9914-cb59364b9ff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:45:59.326572 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594382a0-6646-46b4-9914-cb59364b9ff9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:45:59.326859 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594382a0-6646-46b4-9914-cb59364b9ff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:45:59.327164 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594382a0-6646-46b4-9914-cb59364b9ff9 admin admin] PATH_INFO: `/v3/projects/c9234311754747439023d17adb301608` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:45:59.338315 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594382a0-6646-46b4-9914-cb59364b9ff9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9234311754747439023d17adb301608)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:45:59.340238 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594382a0-6646-46b4-9914-cb59364b9ff9 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:45:59.386372 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-594382a0-6646-46b4-9914-cb59364b9ff9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9234311754747439023d17adb301608', 'request_id': 'req-594382a0-6646-46b4-9914-cb59364b9ff9'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:45:59.392150 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-594382a0-6646-46b4-9914-cb59364b9ff9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9234311754747439023d17adb301608', 'request_id': 'req-594382a0-6646-46b4-9914-cb59364b9ff9'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:45:59.399146 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-594382a0-6646-46b4-9914-cb59364b9ff9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9234311754747439023d17adb301608', 'request_id': 'req-594382a0-6646-46b4-9914-cb59364b9ff9'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:45:59.412078 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-594382a0-6646-46b4-9914-cb59364b9ff9 admin admin] The token cache is being invalidate because project c9234311754747439023d17adb301608 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80154) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:45:59.412405 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-594382a0-6646-46b4-9914-cb59364b9ff9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:45:59.413026 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-594382a0-6646-46b4-9914-cb59364b9ff9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:45:59.414533 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1481/2960] 10.222.0.64 () {58 vars in 1233 bytes} [Tue Aug 2 10:45:59 2022] DELETE /identity/v3/projects/c9234311754747439023d17adb301608 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:46:09.060176 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=7,keystone:DELETE=2 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:46:09.409056 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=44,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:47:15.903372 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f39f762-b8a2-47d5-99b9-41a555e8eb55 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:47:15.904821 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-7f39f762-b8a2-47d5-99b9-41a555e8eb55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:47:15.962109 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f39f762-b8a2-47d5-99b9-41a555e8eb55 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:47:16.011205 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f7350e-3ffb-4a5d-a6bc-ddb20617feaf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:47:16.011815 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f7350e-3ffb-4a5d-a6bc-ddb20617feaf service neutron] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:47:16.012100 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f7350e-3ffb-4a5d-a6bc-ddb20617feaf service neutron] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:47:16.012415 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f7350e-3ffb-4a5d-a6bc-ddb20617feaf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:47:16.014167 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-f9f7350e-3ffb-4a5d-a6bc-ddb20617feaf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:47:16.094596 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f7350e-3ffb-4a5d-a6bc-ddb20617feaf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:47:16.096127 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f7350e-3ffb-4a5d-a6bc-ddb20617feaf service neutron] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:47:16.107073 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1480/2961] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:47:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:47:26.102128 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=79 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:47:33.998569 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f34be5-e911-4b5f-bdd3-f62e246b9ae2 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:47:33.999164 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f34be5-e911-4b5f-bdd3-f62e246b9ae2 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:47:33.999315 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f34be5-e911-4b5f-bdd3-f62e246b9ae2 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:47:34.000940 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1482/2962] 10.222.0.64 () {54 vars in 921 bytes} [Tue Aug 2 10:47:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:47:34.007444 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b53f7a5-0c08-41bb-a3d8-669d212d04bd None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:47:34.007977 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b53f7a5-0c08-41bb-a3d8-669d212d04bd None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:47:34.007977 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b53f7a5-0c08-41bb-a3d8-669d212d04bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:47:34.058239 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-1b53f7a5-0c08-41bb-a3d8-669d212d04bd None None] MFA Rules not processed for user `ec31b188970947efb2ed1d4d9a31cadf`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:47:34.059445 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-1b53f7a5-0c08-41bb-a3d8-669d212d04bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:47:34.066871 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1481/2963] 10.222.0.64 () {58 vars in 1018 bytes} [Tue Aug 2 10:47:34 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:47:34.081980 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f34be5-e911-4b5f-bdd3-f62e246b9ae2 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:47:34.083834 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-24f34be5-e911-4b5f-bdd3-f62e246b9ae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:47:34.098062 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f34be5-e911-4b5f-bdd3-f62e246b9ae2 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:47:34.102005 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82266457-c44b-4f90-8be8-ff7cdc57579e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:47:34.102775 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82266457-c44b-4f90-8be8-ff7cdc57579e service neutron] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:47:34.103116 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82266457-c44b-4f90-8be8-ff7cdc57579e service neutron] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:47:34.103402 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82266457-c44b-4f90-8be8-ff7cdc57579e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:47:34.113116 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82266457-c44b-4f90-8be8-ff7cdc57579e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:47:34.114557 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82266457-c44b-4f90-8be8-ff7cdc57579e service neutron] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:47:34.122924 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1483/2964] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:47:34 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:47:34.207015 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8399bbfe-537e-4308-afc7-eec53430c749 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:47:34.207305 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8399bbfe-537e-4308-afc7-eec53430c749 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:47:34.207585 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8399bbfe-537e-4308-afc7-eec53430c749 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:47:34.208980 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1482/2965] 10.222.0.64 () {54 vars in 921 bytes} [Tue Aug 2 10:47:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 10:47:34.214851 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf91704-2579-4556-9012-9a31ce6c4a92 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:47:34.215109 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf91704-2579-4556-9012-9a31ce6c4a92 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:47:34.215410 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf91704-2579-4556-9012-9a31ce6c4a92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:47:34.255681 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-2bf91704-2579-4556-9012-9a31ce6c4a92 None None] MFA Rules not processed for user `99655e1f9ca24472a86cd26b0dbd1fc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:47:34.256950 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-2bf91704-2579-4556-9012-9a31ce6c4a92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:47:34.266710 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1484/2966] 10.222.0.64 () {58 vars in 1018 bytes} [Tue Aug 2 10:47:34 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:47:34.278223 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8399bbfe-537e-4308-afc7-eec53430c749 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:47:34.290307 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8399bbfe-537e-4308-afc7-eec53430c749 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:47:34.294054 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6833f46-7896-4d04-95c3-3fadbc1016ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:47:34.294682 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6833f46-7896-4d04-95c3-3fadbc1016ba service neutron] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:47:34.294970 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6833f46-7896-4d04-95c3-3fadbc1016ba service neutron] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:47:34.295250 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6833f46-7896-4d04-95c3-3fadbc1016ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:47:34.306191 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6833f46-7896-4d04-95c3-3fadbc1016ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:47:34.307639 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6833f46-7896-4d04-95c3-3fadbc1016ba service neutron] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:47:34.319498 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1483/2967] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:47:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:47:44.253144 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=15 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:47:44.313940 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=19 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:48:17.025650 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf91704-2579-4556-9012-9a31ce6c4a92 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:48:17.027045 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-2bf91704-2579-4556-9012-9a31ce6c4a92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:48:17.120917 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf91704-2579-4556-9012-9a31ce6c4a92 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:48:17.126031 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35550be6-06d3-41e1-8e0f-509cb1e67ce3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b770f073c86b4d88837a8b1f796a8b1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:48:17.126648 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35550be6-06d3-41e1-8e0f-509cb1e67ce3 service placement] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:48:17.126934 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35550be6-06d3-41e1-8e0f-509cb1e67ce3 service placement] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:48:17.127211 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35550be6-06d3-41e1-8e0f-509cb1e67ce3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:48:17.129449 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-35550be6-06d3-41e1-8e0f-509cb1e67ce3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:48:17.140989 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35550be6-06d3-41e1-8e0f-509cb1e67ce3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:48:17.142561 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35550be6-06d3-41e1-8e0f-509cb1e67ce3 service placement] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:48:17.152911 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1485/2968] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:48:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:48:27.148865 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=43 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:49:16.875142 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0bc44c-5bf0-4995-ade2-f53a27e722d6 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:49:16.875786 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0bc44c-5bf0-4995-ade2-f53a27e722d6 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:49:16.875786 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0bc44c-5bf0-4995-ade2-f53a27e722d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:49:16.925907 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-4b0bc44c-5bf0-4995-ade2-f53a27e722d6 None None] MFA Rules not processed for user `ec31b188970947efb2ed1d4d9a31cadf`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:49:16.927200 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-4b0bc44c-5bf0-4995-ade2-f53a27e722d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:49:16.934406 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1484/2969] 10.222.0.64 () {58 vars in 1062 bytes} [Tue Aug 2 10:49:16 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:49:16.940803 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35550be6-06d3-41e1-8e0f-509cb1e67ce3 service placement] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:49:16.951742 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35550be6-06d3-41e1-8e0f-509cb1e67ce3 service placement] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:49:16.956309 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882299c7-a9e3-48ed-be93-2a8a3dcaedbb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:49:16.956959 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882299c7-a9e3-48ed-be93-2a8a3dcaedbb service neutron] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:49:16.957248 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882299c7-a9e3-48ed-be93-2a8a3dcaedbb service neutron] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:49:16.957533 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882299c7-a9e3-48ed-be93-2a8a3dcaedbb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:49:16.959706 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-882299c7-a9e3-48ed-be93-2a8a3dcaedbb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:49:17.038861 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882299c7-a9e3-48ed-be93-2a8a3dcaedbb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:49:17.040875 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882299c7-a9e3-48ed-be93-2a8a3dcaedbb service neutron] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:49:17.055051 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1486/2970] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:49:16 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:49:20.521227 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0bc44c-5bf0-4995-ade2-f53a27e722d6 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:49:20.530567 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0bc44c-5bf0-4995-ade2-f53a27e722d6 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:49:20.535381 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add9284c-8113-458f-9234-fe41cc1ca19e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:49:20.536030 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add9284c-8113-458f-9234-fe41cc1ca19e service neutron] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:49:20.536353 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add9284c-8113-458f-9234-fe41cc1ca19e service neutron] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:49:20.536637 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add9284c-8113-458f-9234-fe41cc1ca19e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:49:20.538715 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-add9284c-8113-458f-9234-fe41cc1ca19e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:49:20.555898 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add9284c-8113-458f-9234-fe41cc1ca19e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:49:20.558243 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add9284c-8113-458f-9234-fe41cc1ca19e service neutron] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:49:20.571873 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1485/2971] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:49:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:49:23.243077 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882299c7-a9e3-48ed-be93-2a8a3dcaedbb service neutron] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:49:23.258183 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882299c7-a9e3-48ed-be93-2a8a3dcaedbb service neutron] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:49:23.264556 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71f554d-453d-46f1-a6bf-eef7c29dd583 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec31b188970947efb2ed1d4d9a31cadf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:49:23.265619 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71f554d-453d-46f1-a6bf-eef7c29dd583 service neutron] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:49:23.266131 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71f554d-453d-46f1-a6bf-eef7c29dd583 service neutron] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:49:23.266626 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71f554d-453d-46f1-a6bf-eef7c29dd583 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:49:23.269713 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-a71f554d-453d-46f1-a6bf-eef7c29dd583 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:49:23.393078 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71f554d-453d-46f1-a6bf-eef7c29dd583 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:49:23.394484 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71f554d-453d-46f1-a6bf-eef7c29dd583 service neutron] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:49:23.418003 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1487/2972] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:49:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:49:27.357111 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add9284c-8113-458f-9234-fe41cc1ca19e service neutron] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:49:27.360580 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-add9284c-8113-458f-9234-fe41cc1ca19e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:49:27.392763 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add9284c-8113-458f-9234-fe41cc1ca19e service neutron] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:49:27.405315 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d38f95f-f50d-4fb3-8cee-b7fed30d5a96 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b770f073c86b4d88837a8b1f796a8b1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:49:27.406387 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d38f95f-f50d-4fb3-8cee-b7fed30d5a96 service placement] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:49:27.406895 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d38f95f-f50d-4fb3-8cee-b7fed30d5a96 service placement] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:49:27.407408 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d38f95f-f50d-4fb3-8cee-b7fed30d5a96 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:49:27.411763 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-6d38f95f-f50d-4fb3-8cee-b7fed30d5a96 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:49:27.431429 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d38f95f-f50d-4fb3-8cee-b7fed30d5a96 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:49:27.441683 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d38f95f-f50d-4fb3-8cee-b7fed30d5a96 service placement] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:49:27.462269 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1486/2973] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:49:27 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:49:31.811296 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71f554d-453d-46f1-a6bf-eef7c29dd583 service neutron] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:49:31.833485 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71f554d-453d-46f1-a6bf-eef7c29dd583 service neutron] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:49:31.847181 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea87f0c-035e-46c5-ab02-130d557c412b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b770f073c86b4d88837a8b1f796a8b1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:49:31.847847 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea87f0c-035e-46c5-ab02-130d557c412b service placement] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:49:31.848137 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea87f0c-035e-46c5-ab02-130d557c412b service placement] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:49:31.848456 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea87f0c-035e-46c5-ab02-130d557c412b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:49:31.853443 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-5ea87f0c-035e-46c5-ab02-130d557c412b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:49:31.876368 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea87f0c-035e-46c5-ab02-130d557c412b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:49:31.879334 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea87f0c-035e-46c5-ab02-130d557c412b service placement] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:49:31.904394 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1488/2974] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:49:31 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:49:31.916691 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d38f95f-f50d-4fb3-8cee-b7fed30d5a96 service placement] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:49:31.919828 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-6d38f95f-f50d-4fb3-8cee-b7fed30d5a96 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:49:31.943009 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d38f95f-f50d-4fb3-8cee-b7fed30d5a96 service placement] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:49:31.951316 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07af691-9a38-4c88-8d4f-ce2da82d51a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:49:31.952287 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07af691-9a38-4c88-8d4f-ce2da82d51a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:49:31.952679 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07af691-9a38-4c88-8d4f-ce2da82d51a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:49:31.953239 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07af691-9a38-4c88-8d4f-ce2da82d51a6 admin admin] PATH_INFO: `/v3/users/672442a55e9b47ccb8da313f8d3f4428` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:49:31.956730 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07af691-9a38-4c88-8d4f-ce2da82d51a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=672442a55e9b47ccb8da313f8d3f4428)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:49:31.958970 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07af691-9a38-4c88-8d4f-ce2da82d51a6 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:49:32.018358 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-c07af691-9a38-4c88-8d4f-ce2da82d51a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '672442a55e9b47ccb8da313f8d3f4428'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:49:32.027343 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-c07af691-9a38-4c88-8d4f-ce2da82d51a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '672442a55e9b47ccb8da313f8d3f4428'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:49:32.038108 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-c07af691-9a38-4c88-8d4f-ce2da82d51a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '672442a55e9b47ccb8da313f8d3f4428'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:49:32.038922 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-c07af691-9a38-4c88-8d4f-ce2da82d51a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '672442a55e9b47ccb8da313f8d3f4428'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:49:32.039676 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-c07af691-9a38-4c88-8d4f-ce2da82d51a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '672442a55e9b47ccb8da313f8d3f4428'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:49:32.061691 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1487/2975] 10.222.0.64 () {58 vars in 1224 bytes} [Tue Aug 2 10:49:31 2022] DELETE /identity/v3/users/672442a55e9b47ccb8da313f8d3f4428 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:49:32.282943 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea87f0c-035e-46c5-ab02-130d557c412b service placement] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:49:32.297486 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea87f0c-035e-46c5-ab02-130d557c412b service placement] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:49:32.359440 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:49:32.371340 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243ed1fa-5c33-4868-9290-41acd9acfd9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:49:32.372063 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243ed1fa-5c33-4868-9290-41acd9acfd9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:49:32.372395 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243ed1fa-5c33-4868-9290-41acd9acfd9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:49:32.372773 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243ed1fa-5c33-4868-9290-41acd9acfd9b admin admin] PATH_INFO: `/v3/projects/91f790e57c874588b0e46e22d00afe3e` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:49:32.374062 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243ed1fa-5c33-4868-9290-41acd9acfd9b admin admin] RBAC: Authorizing `identity:delete_project(project_id=91f790e57c874588b0e46e22d00afe3e)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:49:32.376178 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243ed1fa-5c33-4868-9290-41acd9acfd9b admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:49:32.440278 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-243ed1fa-5c33-4868-9290-41acd9acfd9b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91f790e57c874588b0e46e22d00afe3e', 'request_id': 'req-243ed1fa-5c33-4868-9290-41acd9acfd9b'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:49:32.448594 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-243ed1fa-5c33-4868-9290-41acd9acfd9b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91f790e57c874588b0e46e22d00afe3e', 'request_id': 'req-243ed1fa-5c33-4868-9290-41acd9acfd9b'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:49:32.457762 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-243ed1fa-5c33-4868-9290-41acd9acfd9b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91f790e57c874588b0e46e22d00afe3e', 'request_id': 'req-243ed1fa-5c33-4868-9290-41acd9acfd9b'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:49:32.475165 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-243ed1fa-5c33-4868-9290-41acd9acfd9b admin admin] The token cache is being invalidate because project 91f790e57c874588b0e46e22d00afe3e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80154) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:49:32.475685 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-243ed1fa-5c33-4868-9290-41acd9acfd9b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:49:32.476795 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-243ed1fa-5c33-4868-9290-41acd9acfd9b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:49:32.480014 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1489/2976] 10.222.0.64 () {58 vars in 1233 bytes} [Tue Aug 2 10:49:32 2022] DELETE /identity/v3/projects/91f790e57c874588b0e46e22d00afe3e => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:49:32.485328 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07af691-9a38-4c88-8d4f-ce2da82d51a6 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:49:32.488760 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-c07af691-9a38-4c88-8d4f-ce2da82d51a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:49:32.508381 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07af691-9a38-4c88-8d4f-ce2da82d51a6 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:49:32.586393 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77be2662-e6f2-43ec-a255-af4e5f03580c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:49:32.587052 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77be2662-e6f2-43ec-a255-af4e5f03580c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:49:32.587343 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77be2662-e6f2-43ec-a255-af4e5f03580c admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:49:32.587628 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77be2662-e6f2-43ec-a255-af4e5f03580c admin admin] PATH_INFO: `/v3/users/2dce115b92204af5aeacba8b4257ae7c` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:49:32.588912 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77be2662-e6f2-43ec-a255-af4e5f03580c admin admin] RBAC: Authorizing `identity:delete_user(user_id=2dce115b92204af5aeacba8b4257ae7c)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:49:32.590897 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77be2662-e6f2-43ec-a255-af4e5f03580c admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:49:32.616622 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:49:32.649392 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-77be2662-e6f2-43ec-a255-af4e5f03580c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2dce115b92204af5aeacba8b4257ae7c'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:49:32.656881 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-77be2662-e6f2-43ec-a255-af4e5f03580c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2dce115b92204af5aeacba8b4257ae7c'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:49:32.665205 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-77be2662-e6f2-43ec-a255-af4e5f03580c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2dce115b92204af5aeacba8b4257ae7c'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:49:32.665921 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-77be2662-e6f2-43ec-a255-af4e5f03580c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2dce115b92204af5aeacba8b4257ae7c'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:49:32.666536 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-77be2662-e6f2-43ec-a255-af4e5f03580c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2dce115b92204af5aeacba8b4257ae7c'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:49:32.687297 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1488/2977] 10.222.0.64 () {58 vars in 1224 bytes} [Tue Aug 2 10:49:32 2022] DELETE /identity/v3/users/2dce115b92204af5aeacba8b4257ae7c => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:49:32.942502 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243ed1fa-5c33-4868-9290-41acd9acfd9b admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:49:32.944165 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-243ed1fa-5c33-4868-9290-41acd9acfd9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:49:32.960421 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243ed1fa-5c33-4868-9290-41acd9acfd9b admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:49:33.040062 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27872e62-316c-4542-b26b-0aedf811d7d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5f9105c00b4d00a6249c76315d5e3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34e3e6719e2a412f926ff9cdc426bf39', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:49:33.040752 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27872e62-316c-4542-b26b-0aedf811d7d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:49:33.041052 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27872e62-316c-4542-b26b-0aedf811d7d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:49:33.041334 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27872e62-316c-4542-b26b-0aedf811d7d9 admin admin] PATH_INFO: `/v3/projects/bbb19e16311a4aecac84d389a602e4ad` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:49:33.042664 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27872e62-316c-4542-b26b-0aedf811d7d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bbb19e16311a4aecac84d389a602e4ad)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:49:33.044780 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27872e62-316c-4542-b26b-0aedf811d7d9 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:49:33.101629 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-27872e62-316c-4542-b26b-0aedf811d7d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bbb19e16311a4aecac84d389a602e4ad', 'request_id': 'req-27872e62-316c-4542-b26b-0aedf811d7d9'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:49:33.109279 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-27872e62-316c-4542-b26b-0aedf811d7d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bbb19e16311a4aecac84d389a602e4ad', 'request_id': 'req-27872e62-316c-4542-b26b-0aedf811d7d9'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:49:33.116682 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-27872e62-316c-4542-b26b-0aedf811d7d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bbb19e16311a4aecac84d389a602e4ad', 'request_id': 'req-27872e62-316c-4542-b26b-0aedf811d7d9'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:49:33.134302 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-27872e62-316c-4542-b26b-0aedf811d7d9 admin admin] The token cache is being invalidate because project bbb19e16311a4aecac84d389a602e4ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80154) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:49:33.134812 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-27872e62-316c-4542-b26b-0aedf811d7d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:49:33.135967 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-27872e62-316c-4542-b26b-0aedf811d7d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:49:33.138700 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1490/2978] 10.222.0.64 () {58 vars in 1233 bytes} [Tue Aug 2 10:49:32 2022] DELETE /identity/v3/projects/bbb19e16311a4aecac84d389a602e4ad => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:49:42.683511 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:DELETE=7,keystone:SELECT=20,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:49:43.128868 dv-cinder-843277-8-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=79,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:53:24.027554 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77be2662-e6f2-43ec-a255-af4e5f03580c admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:53:24.029230 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-77be2662-e6f2-43ec-a255-af4e5f03580c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:53:24.159810 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77be2662-e6f2-43ec-a255-af4e5f03580c admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:53:24.231134 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d67e27-2352-4466-99dc-c1b1d545f518 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b770f073c86b4d88837a8b1f796a8b1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '390bd7c4f91c4465b0c7ab0f9e436c4d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:53:24.232241 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d67e27-2352-4466-99dc-c1b1d545f518 service placement] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:53:24.233284 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d67e27-2352-4466-99dc-c1b1d545f518 service placement] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:53:24.233780 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d67e27-2352-4466-99dc-c1b1d545f518 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:53:24.237008 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-03d67e27-2352-4466-99dc-c1b1d545f518 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:53:24.253326 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d67e27-2352-4466-99dc-c1b1d545f518 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:53:24.255046 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d67e27-2352-4466-99dc-c1b1d545f518 service placement] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:53:24.266662 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1489/2979] 10.222.0.64 () {60 vars in 1365 bytes} [Tue Aug 2 10:53:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:53:34.261723 dv-cinder-843277-8-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=74 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}